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Executive  Summary 

A  PLAN  FOR  THE  APPLICATION  OF  ARTIFICIAL 
INTELLIGENCE  TO  DoD  LOGISTICS 


As  U.S.  weapon  systems  and  the  logistics  infrastructure  become  more  complex, 
they  require  operations  and  maintenance  personnel  with  greater  skills  and  better 
training.  Unfortunately,  the  pool  of  such  qualified  personnel  is  dwindling  at  the 
same  time  as  the  demand  increases. 

The  shortage  of  skilled  maintenance  and  operations  technicians  is  not  limited  to 
the  Department  of  Defense.  United  States  private  industry  is  faced  with  the  same 
problem.  Many  U.S.  companies  are  turning  to  new  technologies  to  increase  the 
quality  of  their  products  and  the  productivity  of  their  work  forces.  Those  same 
technologies  are  available  to  the  Military  Departments  and  Defense  agencies. 

One  of  the  most  significant  of  these  technologies  is  artificial  intelligence  (AJ). 
Under  development  for  the  past  30  years,  AI  is  a  family  of  technologies  that  use 
stored  expertise  and  knowledge.  With  these  technologies,  knowledge,  a  vital 
corporate  resource,  can  be  captured  and  tested,  improved  and  replicated,  used  and 
reused. 

Six  AI  technologies  have  significant  potential  for  application  to  production  and 
logistics  processes:  expert/knowiedge-based  systems,  natural  language,  speech 
recognition,  three-dimensional  vision,  intelligent  robotics,  and  neural  networks.  The 
most  significant  of  these  technologies  thus  far  is  expert  systems.  Many  private-sector 
and  DoD  applications  have  shown  that  expert  systems  can  enhance  the  quality  and 
productivity  of  logistics  systems  and  processes. 

The  largest  corporate  users  of  AI  have  recognized  that  productivity-enhancing 
applications  of  AI  exist  throughout  their  corporations  at  virtually  every  level  and  in 
every  function.  However,  they  have  also  recognized  that  AT  will  not  be  applied 
correctly  without  some  formal,  centralized  management  and  support.  Thus,  these 
corporations  created  program  offices  to  encourage,  support,  and  help  manage  the 
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infusion  of  AJ  throughout  their  enterprises.  Their  experience  has  been  highly 
positive  and  provides  a  model  for  DoD. 

The  Assistant  Secretary  of  Defense  (Production  and  Logistics)  [ASD(P&L)]  has 
established  an  initiative  to  facilitate  the  use  of  AJ  technologies  in  DoD  production 
and  logistics  processes.  We  recommend  three  organizational  actions  to  successfully 
implement  that  initiative: 

•  OSD  should  establish  a  DoD  Logistics  AJ  Policy  Council  [chaired  by  the 
ASD(P&L)]  to  provide  DoD-wide  guidance  and  policy  direction  on  matters 
affecting  the  application  of  AJ  technologies  to  logistics  processes. 

•  OSD  should  establish  a  DoD  Logistics  Artificial  Intelligence  Program  Office 
for  a  period  not  to  exceed  3  years  with  a  dedicated  six-person  staff.  The 
Program  Office  will  assist  the  AI  Policy  Council  in  implementing  DoD  AJ 
policy  and  in  establishing  a  permanent  DoD  technical  support  staff.  It  will 
also  promote  AI  throughout  DoD  logistics. 

•  OSD  should  establish  a  DoD-wide  Logistics  Artificial  Intelligence 
Technology  Transition  Center  (TTC)  as  a  permanent  group  to  provide  the 
informational,  technical,  and  managerial  support  services  required  by  the 
logistics  community. 

These  recommendations  constitute  a  strategic  plan  designed  to  help  DoD 
agencies  put  AJ  technologies  to  work  in  logistics.  Implementing  this  plan  will 
improve  logistics  efficiency,  reduce  costs,  increase  effectiveness,  and  decrease  the 
time  required  for  planning  and  mobilization. 


VI 


PREFACE 


The  Assistant  Secretary  of  Defense  (Production  and  Logistics)  [ASDCP&L)]  has 
established  an  initiative  to  facilitate  the  implementation  of  artificial  intelligence 
( AI)  technologies  in  production  and  logistics  processes.  Experience  has  shown  that  AI 
technologies,  when  properly  applied,  are  key  enabling  technologies  that  will  allow 
the  Armed  Foices  i-u  suotain  and  enhance  their  critical  levels  of  warfighting 
capability  in  the  environment  of  the  1990s. 

The  plan  presented  here  documents  the  actions  necessary  to  realize  all  the 
potential  benefits  these  technologies  offer  to  logistics  support  and  management  over 
the  period  1990  —  1995.  The  plan  provides  a  roadmap  which  targets  specific 
objectives  and  charts  the  courses  necessary  for  their  attainment. 

This  plan  provides  the  ASD(P&;L)  and  the  Deputy  Assistant  Secretary  of 
Defense  (Logistics)  with  recommendations  for  the  best  use  of  AI  in  improving 
logistics  efficiency,  reducing  costs,  increasing  effectiveness,  and  decreasing  the  time 
required  for  planning  and  mobilization.  It  outlines  the  organizational  structure  and 
policy  needed  to  support  the  use  of  AI  technologies  in  DoD  logistics. 

This  plan  is  described  in  four  chapters  and  five  appendices.  Chapter  1  assesses 
six  key  AI  technologies  that  can  be  applied  directly  to  military  logistics  processes. 
Chapter  2  reviews  and  analyzes  AI  implementation  experience.  In  that  chapter,  we 
present  a  review  of  major  AI  logistics  initiatives  that  have  been  undertaken  by  the 
Services;  we  also  review  the  experiences  of  six  major  American  corporations  that  are 
in  the  process  of  developing  AI  capabilities  for  their  ongoing  operations.  Chapters 
describes  the  need  for  a  DoD  logistics  AI  program,  outlines  the  scope  and  objectives  of 
the  program,  and  summarizes  the  functions  that  such  a  program  should  perform. 
Chapter  4  then  discusses  the  implementation  of  the  DoD  AI  logistics  program.  The 
five  appendices  contain  detailed  information  and  supporting  documentation. 
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CHAPTER  I 


ARTIFICIAL  INTELLIGENCE  AND  LOGISTICS 

MILITARY  LOGISTICS  IN  THE  1990s 

In  the  1990s,  several  factors  will  combine  to  significantly  change  the  practice  of 
logistics  management.  Those  factors  include  the  ever-increasing  sophistication  of 
our  weapon  systems  and  the  shrinking  pool  of  available,  qualified  manpower. 
Further  complicating  the  problem,  a  growing  percentage  of  entry  level  workers  will 
not  have  the  educational  background  to  qualify  for  skilled  positions.  Still  others  will 
be  eliminated  by  drug  and  substance  abuse  [1].  The  ability  of  the  u.S.  Military 
Services  to  maintain  existing  levels  of  warfighting  capability  in  this  increasingly 
complex  environment  will  be  challenged. 

Furthermore,  the  very  nature  of  h  gistics  decision  making  is  evolving  and  will 
continue  to  evolve  in  the  1990s.  Computer  hardware  and  software  advances  are 
giving  workers  in  a  variety  of  jobs  nearly  routine  access  to  computer  technology.  As 
computers  and  computer  programs  have  become  more  user  friendly,  the  requirement 
to  be  technically  skilled  in  data  processing  in  order  to  gain  access  to,  and  make  use  of, 
decision  support  tools  and  to  gain  access  to  the  organization’s  databases  has  nearly 
been  eliminated.  The  increasing  use  of  both  computer-aided  acquisition  and  logistic 
support  (CALS)  and  electronic  data  interchange  (EDI)  is  making  data  even  more 
accessible  to  support  the  logistics  decision-making  process.  Hence,  we  have  ready 
access  to  data  (resulting  at  times  in  an  inundation  of  logistics  data)  and  routine 
access  to  computers  that  can  process  the  data.  We  cannot,  however,  translate  these 
data  to  useful  information  and  then  ultimately  use  that  information  to  improve 
knowledge  and  understanding  of  the  logistics  process,  and  without  that  capability  the 
data  and  computers  cannot  be  effective. 

Artificial  intelligence  ( AI)  technologies  appear  particularly  promising  as  tools 
to  provide  the  bridge  between  data  and  knowledge.  AI  technologies  permit 
organizations  to  capture  and  use,  in  entirely  new  ways,  the  organization's  most 
important  resource  —  its  corporate  knowledge.  Knowledge  is  now  recognized  as  a 
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vital  corporate  resource,  a  resource  to  be  captured  and  tested,  improved  and 
replicated,  used  and  reused. 

ASSESSMENT  OF  ARTIFICIAL  INTELLIGENCE  TECHNOLOGIES 

Artificial  intelligence  generally  describes  softv/are  that  emulates  some  aspects 
of  human  reasoning.  Much  as  its  name  implies,  it  is  concerned  w^th  developing 
computer  programs  to  make  computers  smart  or  "intelligent.”  It  is  not  one  but  a 
series  of  technologies  that  have  been  in  constant  development  for  the  past  30  years. 
Those  technologies  include  expert  systems,  natural  language,  speech  recognition, 
three-dimensional  vision,  intelligent  robotics,  and  neural  networks.  AI  is  based  on 
the  premise  that  common  processes  underlie  the  activities  of  thought  and  perception 
and  these  processes  can  be  understood  and  functionally  replicated  in  computers. 
However,  the  reasoning  processes  with  which  AI  concerns  itself  are  symbolic  and 
nonalgorithmic  rather  than  numeric.  Most  of  the  interesting  knowledge  that  people 
possess  is,  in  fact,  symbolic,  inexact  knowledge  [2].  AI  attempts  to  understand  and 
represent  this  symbolic,  inexact  knowledge  so  that  it  can  be  processed  by  th^: 
computer. 

Artificial  intelligence,  however,  is  not  a  revolutionary  technology.  Rather,  it 
represents  an  evolutionary  step  forward  in  the  representation  of  symbolic  knowledge. 
Important  distinctions  can  be  drawn  between  conventional  computer  programming 
and  AI.  In  conventional  computer  programs,  the  instructions  are  explicit  and  define 
precise  solution  algorithms.  On  the  other  hand,  AI  programs  do  not  explicitly  define 
solution  steps.  They  define  specific  control  strategies,  such  as  working  backward 
from  a  goal  to  a  solution  or  working  forward  from  data  to  a  solution.  The  computer 
then  follows  these  strategies  to  generate  a  solution,  much  as  an  individual  uses  logic 
to  solve  a  problem. 

An  AI  logistics  program  captures  and  stores  logistics  knowledge,  such  as  rules, 
policies,  and  logic,  in  a  "knowledge  base”  in  much  the  sami-  way  as  a  conventional 
computer  program  stores  numeric  information  in  a  database.  However,  while 
conventional  computer  programs  integrate  the  domain  knowledge  and  the  control 
logic,  so  that  it  is  difficult  to  modify  the  program  or  to  separate  the  knowledge  from 
the  control  structure,  AI  programs  clearly  separate  the  domain  knowledge  from  the 
control  logic.  (Figure  I- 1  illustrates  that  separation).  Because  of  the  separation,  AI 
programs  tend  to  be  easier  to  modify,  update,  and  enlarge.  AI  programs  can  also 
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work  with  uncertain,  conflicting,  or  missing  facts,  and  explain  the  logic  of  a 
suggested  solution.  Figure  1-2  is  a  comparison  of  conventional  and  symbolic  A 
programming;  it  lists  the  strengths  of  each  technique.  Conventional  programming 
and  s^Tnbolic  programming  can  and  should  be  used  together  in  solving  complex 
logistics  problems;  each  technique  should  be  used  to  address  the  types  of  tasks  for 
which  it  is  best  suited. 


Inference  engine 

Knows  how  to  reason  about  knowledge 
Knows  how  to  explain  its  reasoning 
Knows  how  to  help  build  correct,  consistent 
expert  systems 


I  Knowledge  base 

!  Knowledge  IS  represented  in  easily 

I  understood  rules  or  frames.  Order 

I  IS  usually  not  important. 

I _  _ ^ _  _ 

t  t 

This  IS  the  expert  system  The  programmer  needs  to 

building  tool  or  shell  that  is  write/build  only  this  part  of 

provided  (already  built)  to  the  the  system, 

expert  system  programmer. 


FIG.  1-1.  ARTIFICIAL  INTELLIGENCE  SYSTEM  ARCHITECTURE 


Conventional  programming 

•  Algorithms 

•  Numerically  addressed  database 

•  Oriented  toward  numerical  processing 

•  Sequential  batch  processing 

•  Mid-run  explanation  impossible 


Symbolic  programming 

•  Heuristics 

•  Symbolically  structured  knowledge 
base  in  a  global  working  memory 

•  Oriented  toward  symbolic  processing 

•  Highly  interactive 

•  Mid-run  explanation  easy 
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FIG.  1-2.  COMPARISON  OF  CONVENTIONAL  AND  KNOWLEDGE-BASED  SYSTEMS 
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Six  AJ  technologies  have  significant  potential  for  application  to  production  and 
logistics  processes;  expert/ knowledge-based  systems,  natural  language,  speech 
recognition,  three-dimensional  vision,  intelligent  robotics,  and  neural  networks. 
Each  of  these  technologies  is  discussed  in  this  chapter.  In  those  discussions,  we 
present  a  brief  description  of  the  technology,  an  assessment  of  the  technology’s 
current  status  and  future  potential,  and  actual  or  proposed  applications  of  the  specific 
technology  to  DoD  production  and  logistics  processes. 

Expert  Systems 

Expert  systems  are  software  programs  designed  to  mimic  a  human  expert.  The 
programs  usually  serve  as  intelligent  advisors  in  specific  areas  of  expertise.  Many 
management  decisions  in  logistics  functional  areas  are  made  under  complex 
circumstances  by  acquiring  information  about  particular  problems  and  then  applying 
"rules  of  thumb,"  facts,  heuristics,  and  other  knowledge  obtained  through  experience. 
A  "knowledge  engineer”  may  use  a  process  known  as  "cognitive  modeling,”  to  encode 
the  knowledge  of  the  domain  and  the  way  the  expert  acquires,  represents,  and  uses 
that  knowledge.  Expert  systems  capture  the  knowledge  in  the  form  of  a  computerized 
knowledge  base.  The  expert  system  then  uses  the  stored  knowledge  along  with 
problem-specific  information  to  suggest  expert  solutions.  Expert  systems  expand  the 
types  of  decision  making  that  can  be  enhanced  through  automation. 

To  aid  a  manager  in  resolving  a  problem  in  logistics,  an  expert  system  may 
request  some  initial  information  about  the  problem  from  the  user.  It  then  searches 
the  knowledge  base  for  an  entry  (a  rule,  pattern,  or  model)  or  entries  that  "fit"  the 
problem  situation  suggested  by  the  initial  data.  This  search  may  lead  to  an 
immeuiace  solution.  More  likely,  it  will  result  in  a  request  for  additional 
information.  That  process  continues  until  the  expert  system  reaches  a  conclusion  or  a 
recommended  management  action.  At  any  time  during  the  process,  a  manager  may 
ask  the  expert  system  to  explain  a  line  of  questioning,  the  reasoning  process  that  led 
to  a  particular  conclusion,  or  the  relevance  of  a  recommended  action. 

Most  expert  systems  can  offer  advice,  solve  problems,  or  make  predictions  with 
an  accuracy  almost  as  great  as  their  human  counterparts  and  in  rare  cases  even 
better.  Expert  systems  usually  are  interactive;  however,  they  can  also  be  used  to 
control  processes  in  a  closed  loop  real-time  environment,  with  no  human 
intervention. 
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Expert  system  technology  has  some  technical  restrictions.  Currently,  expert 
systems  can  address  only  very  narrow  areas  of  expertise  and  have  limited  capability 
to  encode  common  sense.  Current  research  efforts  are  aimed  at  expanding  their 
limited  explanation  and  learning  capabilities.  The  next  5  years  should  witness  the 
development  of  enhanced  schemas  for  representing  knowledge,  the  ability  to  provide 
much  better  explanations,  much  larger  and  more  sophisticated  knowledge  bases,  and 
knowledge  base  sharing  among  numerous  systems.  The  ability  to  incorporate  true 
common  sense  and  learning  features  will  remain  a  difficult  problem.  Further,  useful 
programming  techniques  such  as  object-oriented  programming,  model  based 
reasoning,  and  generic  task  tools  will  be  advanced  and  applied  in  new  areas. 

The  ability  of  expert  systems  to  encode  institutional  knowledge  has  wide- 
ranging  advantages.  Where  expertise  is  in  short  supply,  expert  systems  help 
increase  productivity  by  enabling  the  sharing  of  critical  skills  at  different  sites  and 
throughout  all  levels  of  the  organization.  Workers  at  all  levels  are  able  to  deal  with 
increasing  levels  of  complexity  because  they  have  access  to  higher  levels  of  expertise. 
The  development  of  expert  systems  also  enables  an  organization  to  assimilate  the 
knowledge  and  experience  of  several  human  experts,  often  from  different  fields. 
Some  systems  are  developed  in  the  hope  of  creating  expertise  in  critical  areas  in 
which  not  enough  knowledge  exists.  The  use  of  expert  systems  promotes  disciplined 
application  of  procedures  and  policies,  and  can  shorten  the  training  time  for  new 
personnel.  Expert  systems  are  also  used  to  provide  expertise  in  boring  or  hazardous 
jobs  that  do  not  retain  nor  attract  experts.  Of  even  greater  import,  expert  systems 
can  provide  more  consistent,  reliable,  timely,  high-quality  decisions. 

Expert  system  technology  has  been  proven.  Worldwide,  more  than 
2,000  expert/knowledge-based  systems  are  in  regular  use  in  the  public  and  the 
private  sectors  [3|.  In  the  private  sector,  wholesalers,  retailers,  and  manufacturers 
use  expert  systems  daily  in  all  functional  areas.  Digital  Equipment  Corporation 
(DEC)  uses  .50  expert  systems  on  a  daily  basis  [4].  Some  of  the  benefits  of  these 
systems  are  difficult  to  quantify,  "but  overall  the  net  return  to  Digital  is  estimated  to 
be  in  excess  of  $40  million  per  year”  [5].  International  Business  Machines  (IBM)  has 
nearly  400  expert  systems  in  use  and  under  development.  It  claims  a  net  return  of 
$.38.5  million  a  year  on  its  six  leading  expert  system  applications  [6].  Du  Pont  has 
more  than  600  expert  systems  in  use  and  claims  annual  returns  of  15  to  1  on  those 
systems  [7]. 


Commercial  Logistics  Applicatior^s 

Many  of  these  expert  system  applications  address  real-world  logistics 
operations.  The  Eastman  Kodak  Company  uses  an  expert  system  in  its  Atlanta  and 
Dallas  distribution  centers  to  greatly  improve  the  productivity  of  its  work  force  in 
picking  material  and  correctly  placing  it  on  pallets.  This  expert  system  is  based 
entirely  on  personal  computers  (PCs)  [8].  It  examines  the  line  items  on  a  customer’s 
order  and  decides  (1)  how  to  build  pallets  of  product  that  are  ready  to  ship  to  the 
customer  and  (2)  how  to  select  the  best  "pick  path”  through  the  warehouse.  To 
develop  the  expert  system,  an  experienced  order  puller’s  thinking  process  was 
captured  and  made  available  to  everyone.  As  a  result,  pick  productivity  was 
increased,  and  the  training  period  for  new  people  was  reduced  from  30  to  60  days 
down  to  1  to  2  days. 

General  Motors  (GM)  uses  expert  systems  for  scheduling  its  manufacturing 
operations  and  for  a  number  of  diagnostic  tasks.  It  uses  an  expert  system  called 
"Charley”  to  analyze  the  vibration  signatures  of  machines  and  to  assist  in  the 
diagnosis  of  machinery  problems.  For  23  years,  Charley  Amble  was  GM’s  resident 
vibration  analysis  expert.  He  diagnosed  real  and  potential  maintenance  problems  on 
plant  floor  machinery  by  analyzing  machine  vibration  signatures.  Because  his 
retirement  in  1986  would  have  meant  not  only  the  loss  of  his  expertise  but  also 
increased  training  costs  and  reduced  levels  of  machine  maintenance,  GM  decided  to 
"clone"  Charley’s  knowledge  of  machine  vibration  signatures.  "Charley”  is  now  in 
use  at  three  GM  plants  in  Michigan  and  when  in  full  production,  will  be  installed  at 
20  of  GM’s  North  American  plants.  At  the  Saginaw  plant  alone,  Charley  has  saved 
approximately  $65,000  by  diagnosing  problems  in  three  different  machines  before 
complete  failure  occurred.  GM  estimates  that  each  machining  plant  that  uses  the 
expert  system  will  see  cost  savings  of  nearly  $500,000  per  year  [9], 

Burlington  Northern  Railroad  uses  an  expert  system  that  identifies  scheduling 
conflicts  among  trains  and  track  maintenance  gangs.  Called  the  Service/ 
Maintenance  Planner,  it  recommends  solutions  to  these  conflicts.  In  scanning  the 
February  and  March  1989  schedules,  for  example,  the  system  identified  solutions 
saving  more  than  $100,000  in  costs  associated  with  train  delays  and  wasted 
maintenance  gang  time  [10]. 
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Digital  Equipment  Corporation  has  been  using  expert  systems  for  logistics 
since  the  early  1980s.  One  of  its  most  successful  applications  is  a  system  called 
"Dispatcher”  which  keeps  track  of  work-in-process  (WTP)  inventories.  Dispatcher  is 
integrated  with  the  company’s  materials  handling  and  supplies  ordering  system.  It 
ensures  WIP  inventory  is  always  being  processed  by  an  available  workstation. 
Further,  it  sees  that  orders  for  supplies  and  component  parts  are  placed  as  soon  as  a 
need  is  identified.  Dispatcher  has  enabled  DEC  to  cut  its  WIP  cycle  times  from 
36  days  to  5  days  [11]. 

Texas  Instruments  uses  expert  systems  to  help  customer  service  represen¬ 
tatives  answer  more  questions  than  would  otherwise  be  possible  [12].  The  Federal 
Mogul  Corporation  uses  an  expert  system  to  assist  in  forecasting  requirements  for 
inventory  [13].  IBM  uses  an  expert  system  called  MOLDCOST  to  assist  in  the 
purchase  of  injection  moldings.  The  system  saves  IBM  $1.5  million  annually  [6]. 
Literally  hundreds  of  additional  logistics  applications  are  in  use  by  companies  as 
diverse  as  American  Airlines,  Mervyns,  Marshalls,  Federal  Express,  DuPont, 
Westinghouse,  and  Ford. 

OoD  Logistics  Applications 

Expert  system  technology  has  also  been  applied  to  all  functional  areas  of 
production  and  logistics  throughout  DoD.  Today,  more  than  400  logistics  expert 
systems  are  under  development  or  in  use  in  areas  of  inventory  management, 
transportation,  warehousing,  acquisition,  maintenance,  and  production.  For 
example,  the  Air  Force  has  fielded  the  Inventory  Manager’s  Assistant,  an  expert 
system  that  validates  the  data  used  in  the  computation  of  reparable  assets 
forecasting.  During  its  first  field  test  the  system  identified  a  $600,000  cost 
savings  [14], 

The  Joint  Chiefs  of  Staff  (JCS)  is  supporting  the  development  of  the  Logistics 
Intra-theater  Support  Tool.  It  gives  advice  on  the  adequacy  of  a  theater’s 
infrastructure  to  support  the  movement  of  specified  ground  forces  and  supplies  to  a 
given  destination  by  a  specified  day  [15]. 

The  Navy  is  currently  fielding  an  expert  system  —  the  Dues  Management 
Advisor  (DMA)  —  that  assists  in  the  management  of  critical  items.  It  was  developed 
to  assist  Navy  inventory  personnel  manage  past-due  replenishment  orders  for  retail 
items  at  a  number  of  supply  centers.  Specifically,  it  identifies  and  sets  priorities  for 
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the  most  critical  past-due  items  for  management  attention  at  any  given  time.  The 
use  of  DMA  is  expected  to  result  in  faster  resolution  of  past-due  situations  and  more 
consistent  application  of  procedures  [16], 

The  Army  is  developing  the  Logistics  Planning  and  Requirements  Simpli¬ 
fication  (LOGPARS)  system.  It  offers  integrated  logistics  support  (ILS)  expertise  to 
assist  in  the  development  of  the  ILS  strategy  and  readiness  objectives  [17].  Each  of 
the  Services  has  several  expert  systems  that  assist  with  the  diagnosis  and  fault 
isolation  of  sophisticated  weapon  systems,  and  numerous  other  expert  systems  are  in 
use  or  under  development. 

Natural  Language 

Natural  language  is  the  second  important  AI  technology.  Natural  language 
systems  can  convert  a  language  such  as  English,  French,  Japanese,  or  German  into 
computer  language.  With  natural  language  systems,  we  can  communicate  with 
computers  in  our  native  language,  thus  eventually  eliminating  the  requirement  to 
learn  computer  languages.  Furthermore,  natural  language  systems  make  our 
logistical  databases  easier  to  access  and  use. 

With  some  limitations,  natural  language  technology  is  commercially  available 
today.  Natural  language  interfaces  are  available  for  both  PCs  and  mainframe 
computers.  The  first  commercially  available  natural  language  systems  for 
mainframes  were  introduced  in  the  early  1980s,  and  PC-based  versions  appeared  in 
the  mid-1980s.  Most  of  the  tools  that  are  available  today  require  either  a  structured 
form  of  query  or  a  specific  relational  database  management  system  interface.  We 
have  not  yet  developed  tools  that  permit  unstructured  queries  of  flat  files. 
Restrictions  on  the  use  of  natural  language  tools  will  be  reduced  and  much  more 
powerful  capabilities  will  continue  to  be  introduced  over  the  next  several  years. 

Natural  language  applications  are  being  examined  for  their  potential  use  in 
DoD.  Among  the  projects  proposed  is  one  that  would  combine  the  use  of  natural 
language  with  an  expert  simulation  to  provide  action  officers  in  the  Pentagon  with  a 
query  capability  to  do  what-if  analyses  on  budget  cuts.  Such  a  system  would  permit 
action  officers  to  type  questions  like  "What  other  programs  will  be  afTected  if  I  cut  the 
fuels  budget  by  $500  million?”  and  receive  a  response.  Numerous  other  agencies 
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throughout  DoD  have  suggested  the  use  of  natural  language  systems  to  provide  user- 
friendly  query  capability  for  their  huge  logistical  databases. 

Speech  Recognition 

Speech  recognition  and  understanding  is  the  third  key  AI  technology.  Speech 
recognition  systems  permit  us  to  talk  directly  to  computers  and  eliminate  the 
requirement  for  keyboard  entry.  All  speech  systems  work  in  essentially  the  same 
way:  An  analog  voice  signal  is  digitized  and  compared  with  stored  patterns  to 
recognize  what  was  said,  and  stored  grammatical  rules  are  applied  to  determine  the 
meaning  conveyed  by  the  words.  The  simplest  analysis  technique  is  template 
matching.  It  consists  of  comparing  the  digitized  input  signal  with  stored  templates  in 
the  system  until  a  match  is  found.  Such  simple  pattern  matching  might  suffice  for 
discrete  speech  (i.e.,  isolated  words  with  a  distinct  pause  between  words)  from  a 
single  user.  However,  speaker-independent  systems  that  must  understand  contin¬ 
uous  speech  require  much  more  sophisticated  recognition  algorithms.  Furthermore, 
as  one  might  expect,  the  larger  the  vocabulary  the  computer  must  recognize,  the 
more  complicated  the  systems  become. 

Commercial  Logistics  Applications 

Speech  recognition  systems  are  commercially  available  today.  However,  the 
types  and  capabilities  of  these  systems  are  still  somewhat  limited.  Current  speech 
recognition  technology  is  largely  dependent  upon  the  requirements  of  the  system  and 
its  environment,  e.g.,  whether  the  system  is  speaker-dependent  or  speaker- 
independent;  whether  the  speech  is  isolated,  connected,  or  continuous;  the  size  of  the 
vocabulary;  the  noise  level  of  the  operating  environment;  and  the  stress  of  the 
operator  whose  speech  must  be  recognized. 

At  the  Ford  Motor  Company,  inspectors  use  a  speech  recognition  system  in  the 
form  of  wireless  microphones  and  headsets,  which  are  tied  into  the  company’s  central 
management  information  system.  As  the  cars  come  down  the  assembly  line,  the 
speech  system  automatically  prompts  the  inspectors  to  respond  to  all  the  items  on  the 
quality  control  checklist.  This  system  ensures  real-time  data  capture  and  also 
ensures  the  inspectors  totally  comply  with  the  appropriate  checklist.  If  an  inspector 
fails  to  respond  to  an  item  on  the  checklist,  the  line  stops.  The  speech  system  uses 
discrete  speech  and  a  limited  vocabulary  to  improve  accuracy.  This  is  an  example  of 
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how  AI  technology  can  be  used  to  implement  the  Total  Quality  Management 
Program. 

DoO  Logistics  Applications 

The  Army’s  Tank  and  Automotive  Command  (TACOM)  is  using  a  speech 
recognition  system  combined  with  an  expert  diagnostic  system  for  the  maintenance 
of  its  general-purpose  motor  vehicles.  Technicians  fixing  the  vehicles  wear  headsets 
equipped  with  wireless  microphones  that  permit  them  to  receive  instructions  from 
the  expert  diagnostic  system  and  to  communicate  with  the  system  while  under  or 
moving  about  the  vehicle  to  check  for  symptoms  or  to  make  repairs. 

The  Air  Force  Logistics  Command  (AFLC)  is  using  a  speech  recognition  system 
in  one  of  its  depot  warehouses  to  interface  with  the  warehouse's  automated  storage 
module  (ASM).  The  warehouseman  (wearing  a  wireless  microphone  and  headset) 
picks  up  an  item  from  a  container  of  incoming  goods  and  reads  the  part  number 
aloud.  The  system  updates  the  inventory  and  provides  instructions  to  the 
warehouseman  to  tell  him  the  row  in  which  to  place  the  item.  The  warehouseman 
places  the  item  in  the  appropriate  container  in  the  row,  and  the  ASM  automatically 
stores  the  item.  These  items  are  not  bar  coded,  and  prior  to  the  implementation  of  the 
speech  system,  the  warehousemen  had  to  manually  key  the  data  into  the  interface  to 
the  ASM.  That  process  led  to  numerous  typing  errors.  Speech  systems  can 
significantly  increase  inventory  accuracy  simply  by  eliminating  typographical  errors 
generated  at  the  keyboard. 

Three-Dimensional  Vision 

Three-dimensional,  or  stereoscopic,  vision  is  an  AI  technology  that  permits  a 
computer  to  "sense”  its  environment,  thereby  classifying  objects  and  avoiding 
obstacles.  As  does  human  eyesight,  stereoscopic  machine  vision  merges  two  pictures 
into  a  single  image  and  calculates  the  depth  by  measuring  the  displacement  of  the 
object  between  the  two  pictures  or  "eyes.” 

In  developing  three-dimensional  vision  systems,  the  major  problem  is  to  bring 
the  images  from  the  two  cameras  or  sensors  into  alignment  quickly  enough  to 
perform  practical  real-world  tasks.  The  algorithms  designed  to  permit  three- 
dimensional  vision  tend  to  be  computationally  expensive,  and  for  that  reason,  few 
applications  of  such  vision  are  practical  today.  Another  real  problem  with 
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three-dimensional  vision  systems  is  that  our  environment  tends  to  be  quite  "noisy.” 
Rarely  do  we  find  an  object  with  clear  unobstructed  edge  lines.  We  are  much  more 
likely  to  find  a  desk  cluttered  with  books  and  papers,  and  that  clutter  interferes  with 
attempts  to  recognize  the  desk  from  stored  patterns  programmed  into  the  computer. 
To  deal  with  this  and  other  problems,  we  can  program  algorithms  as  cooperative 
computations,  in  which  local  operations  are  performed  in  parallel  on  all  parts  of  the 
data. 

Using  very  complex  algorithms  and  taking  advantage  of  parallel  computing,  we 
are  today  at  the  stage  of  commercial  two-and-a-half-dimensional  vision.  Software 
development  tools,  which  permit  users  to  develop  their  own  machine  vision  systems, 
are  commercially  available.  Although  these  tools  run  on  special-purpose  AI 
machines,  they  will  soon  be  ported  to  the  engineering  computer  workstations  built  by 
Sun  Microsystems  and  to  Macintosh  II  computers  built  by  Apple  Computer. 

The  Air  Force  is  investigating  applications  of  three-dimensional  vision  systems 
that  would  permit  maintenance  personnel  to  reverse  engineer  parts  and  collect  data 
for  reprocurement  or  remanufacture  of  those  parts.  The  Navy  is  also  examining  the 
potential  of  three-dimensional  vision  systems  as  part  of  their  Rapid  Acquisition  of 
Manufactured  Parts  (RAMP)  program.  Stereoscopic  vision  is  also  necessary  for  truly 
intelligent  robotic  systems  to  be  used  as  order  pickers  in  the  warehouses  of  the  future. 

Intelligent  Robotics 

Intelligent  robots  are  manipulators,  or  complete  mobile  systems,  that  combine 
expert  systems,  natural  language,  speech  recognition  and  understanding,  and  vision 
systems  to  create  entities  capable  of  flexible  operations  that  humans  might  otherwise 
perform.  Flexibility  is  the  key  requirement.  The  intelligent  robot  must  be  able  to 
respond  flexibly  to  new  and  unique  situations.  For  such  a  system  to  operate 
effectively  in  a  new  environment,  it  must  be  able  to  devise  a  strategy,  sense  any 
obstacles  that  would  prevent  it  from  executing  the  strategy,  and  achieve  the 
objective.  Hence,  the  ability  for  the  intelligent  robot  to  accurately  recognize  patterns 
and  identify  objects  in  real-  or  near  real-time  is  critical. 

Truly  intelligent  robotic  systems  capable  of  both  movement  and  true  three- 
dimensional  vision  are  limited  to  the  research  laboratory  at  this  time.  Although 


significant  progress  is  being  made  in  these  areas,  existing  robotic  systems  are  much 
too  slow  for  real-world  implementation. 

The  Department  of  Defense,  however,  is  investigating  logistics  applications  of 
intelligent  robotics.  In  an  Air  Force  research  project  at  one  of  its  maintenance  depots, 
an  intelligent  manipulator  is  tasked  with  removing  rivets  and  fasteners  from  the 
wings  of  F-16  aircraft.  The  Navy  has  similar  projects  under  way  as  part  of  its  RAMP 
initiative. 

Neural  Networks 

The  final  AI  technology  area  discussed  in  this  plan  is  neural  networks,  an  area 
also  known  variously  as  neural  computing,  neural  networking,  connectionism,  and 
adaptive  systems.  Neural  networks  are  based  on  theoretical  models  of  how  we 
believe  the  human  brain  works.  The  basic  building  blocks  or  cells  of  the  brain, 
neurons,  are  interconnected  by  synapses.  Neural  networks  are  made  up  of  different 
numbers  of  processors,  which  serve  as  neurons.  These  neurons  are  connected  to  many 
other  neighboring  neurons  to  form  a  complex  network  (Figure  1-3).  These  processors 
communicate  with  one  another  across  the  connections  by  providing  signals  that 
activate  or  inhibit  processing  by  those  neurons  or  processors  to  which  they  are  linked 
(Figure  1-4).  Hence,  a  given  neuron  might  simultaneously  receive  inputs  from  many 
other  neurons.  If  the  sum  of  those  inputs  exceeds  a  set  threshold  value,  the  neuron  is 
said  to  "fire.”  That  firing,  in  turn,  causes  the  neuron  to  produce  an  output  signal, 
which  is  transmitted  to  all  the  neurons  with  which  it  is  connected. 

Neural  networks  offer  several  advantages  over  conventional  computing.  First 
is  the  concept  of  adaptive  behavior  or  learning.  Neural  networks  are  also  capable  of 
"associative  recall,”  which  is  the  ability  to  reach  a  conclusion  or  recommend  an  action 
even  when  an  exact  match  to  stored  patterns  cannot  be  found.  The  ability  to 
"discover”  novel  patterns  or  statistically  significant  characteristics  among  data  is 
still  another  desired  attribute  of  neural  networks.  Finally,  because  of  their 
numerous  interconnections  and  massively  simultaneous  processing,  neural  networks 
will,  we  hope,  some  day  be  able  to  quickly  derive  solutions  to  problems  that  involve 
combinatorial  or  exponential  explosion  of  the  number  of  feasible  solutions. 

Neural  network  technology  has  seen  very  limited  practical  application  to  date. 
Today’s  numerous  neural  network  models  differ  from  one  another  in  the  number  of 
neurons  or  nodes  they  contain,  the  way  in  which  those  neurons  are  interconnected. 
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Source:  Copyright  intellogistics,  Inc..  1988 
Note:  ?£  =  processing  element. 

FIG.  1-3.  A  NETWORK  OF  PROCESSING  ELEMENTS 


Other 

elements 


Fan-in 


Fan-out 


Source:  Copyright  intellogistics,  inc  .  1988 

Note:  !  =  input  value;  W  =  input  weight;  A(t)  =  threshold  weight, 
Out(t)  =  output  value 


FIG.  1-4.  A  TYPICAL  PROCESSING  ELEMENT 


and  the  rules  used  to  produce  each  neuron’s  output  signal.  Tools  to  develop  neural 
solutions  are  available  for  both  microcomputers  and  mainframes.  The  next  5  years 
should  witness  a  significant  increase  in  the  capability  of  this  technology  area. 
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Numerous  DoD  agencies  are  experimenting  with  neural  networks.  The  ability 
of  the  technology  to  recognize  patterns  in  huge  databases  is  one  especially  promising 
logistical  application.  The  Air  Force  is  currently  supporting  a  research  initiative  to 
determine  new  information  about  trends  in  engine  failure  by  having  a  neural 
network  analyze  the  Comprehensive  Engine  Management  System  (CEMS)  database. 
The  system  is  designed  to  determine  whether  any  patterns  in  engine  usage  can  help 
forecast  the  life  span  of  the  engine  based  upon  the  circumstances  of  its  usage.  The  Air 
Force  has  also  proposed  to  replace  a  human  inspector  with  a  camera  tied  to  a  neural 
network  to  perform  nondestructive  inspection  of  parts.  The  system  would  be  able  to 
more  accurately  and  consistently  examine  the  parts  to  check  for  stress  and  existing  or 
potential  structural  problems. 
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CHAPTER  2 


ARTIFICIAL  INTELLIGENCE  IMPLEMENTATION  EXPERIENCE 

CURRENT  INDUSTRY  ARTIFICIAL  INTELLIGENCE  PROGRAMS 

Several  major  U.S.  industrial  companies  have  made  "corporate  commitments” 
to  use  AJ  technologies  to  achieve  higher  profits  and  productivity.  As  early  as  1984, 
Senior  Vice  Presidents  and  Chief  Executive  Officers  (CEOs)  at  companies  such  as 
DEC,  IBM,  GM,  Du  Pont,  EMC  Corporation,  Lockheed,  and  Boeing  were  reviewing 
and  approving  recommendations  made  by  corporate  task  forces  to  view  AI 
components  as  strategic  technologies  important  to  their  businesses  and  to  establish 
program  offices  and/or  centers  to  foster  the  use  of  those  technologies.  This  chapter 
examines  the  philosophy,  organizational  approach,  resource  commitments,  and 
lessons  learned  by  some  of  those  companies  during  the  past  5  years.  More  detailed 
information  about  the  AI  involvement  of  these  companies  is  provided  in  Appendix  A. 

Digital  Equipment  Corporation 

Digital  Equipment  Corporation  views  AI  as  an  integral  part  of  its  business 
success.  AI  has  played  a  crucial  integrating  role  within  DEC.  DEC  got  involved  in 
AI  originally  in  1978  because  it  had  a  critical  business  problem  that  could  not  be 
solved  by  any  other  means.  Today,  DEC  uses  AI  to  create  and  manage  what  it  refers 
to  as  the  company’s  "knowledge  network.”  This  network  comprises  cross-functional 
knowledge  and  information  that  permits  DEC  to  concentrate  on  functional  boundary- 
spanning  processes  as  opposed  to  small  functional-specific  tasks.  As  a  result  of  the 
requirement  for  this  cross-functional  flow  of  information  and  knowledge,  the 
majority  of  DEC’s  AI  systems  are  large  systems  that  combine  AI  techniques  with 
conventional  programming  techniques,  models,  and  databases.  Further,  these  AJ 
systems  tre.'^t  the  flow  of  information,  knowledge,  and  materials  within  DEC  across 
multi-functional  boundaries  (see  Figure  2-1),  The  knowledge  systems  and  the 
development  of  these  systems  integrate  the  functional  activities  and  should  result  in 
improved  processes. 

The  use  of  AI  technologies  to  solve  key  strategic  business  problems  is  part  of 
dec’s  corporate  philosophy.  To  ensure  this  philosophy  is  communicated  throughout 


FIG.  2-1.  THE  MATERIALS  FLOW  PROCESS 


the  company,  DEC  has  established  an  AJ  Board  of  Directors  that  reports  directly  to 
the  Senior  Vice  President  for  Commercial  Operations,  and  in  turn,  to  DEC’S  CEO  (see 
Figure  2-2).  DEC’S  AI  Board  of  Directors  is  chaired  by  the  Vice  President  for 
Networks  and  Distributed  Systems  Group  and  its  members  consist  of  DEC  Vice 
Presidents  for  Engineering,  Marketing,  Manufacturing,  Software  Services,  and  Field 
Services.  The  AI  Board  of  Directors  provides  policy  guidance  and  direction  for  DEC’s 
AJ  activities. 


/Vote:  AifC  =  Ar-'  f  c  infePigence  Technologies  Center'  JSTG  =  intelligent  S/stems 
'ecnnoiogy  GrouD,  AiAG  =  AI  ADolications  Grouo.  AiPG  •=  AI  Product  Group.  AiMG  =  Ai 
VarKPtinq  Group 


FIG.  2-2.  DIGITAL  EQUIPMEN"^  CORPORATION'S  AI  ORGANIZATION 

Several  formal  groups  at  DEC  play  important  roles  in  the  implementation  of  AI 
policy  guidance.  Some  of  these  activities  are  centralized,  while  others  are  more 
decentralized.  The  Artificial  Intelligence  Technologies  Center  (AJTC)  was  formed  in 
1984  and  given  a  permanent  mandate  to  provide  central  education  and  training 
services,  systems  development,  systems  validation,  and  technology  transition 
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functions.  The  Configuration  Systems  Development  Group  (CSDG)  is  a  more 
decentralized  activity.  It  reports  to  DEC’s  corporate  manufacturing  operation  and  is 
responsible  for  the  specific  activities  shown  in  Figure  2-3  for  all  of  DEC’S  config¬ 
uration  management  systems,  company-wide.  Individual  units  are  specifically 
charged  with  ensuring  the  integration  of  AJ  with  conventional  systems  and  also 
ensuring  the  appropriate  use  of  the  technology. 


< nowiedge  Enginee''ing- 
'Jew  Systems  Oeve'ODment 


designs  ana  deveioDs  Ai  aspects  of 
new  expert  systems 

6  people 


Knowledge  Engineering: 
Operational  Systems  Development 


Knowledge  engineering  of  exten¬ 
sions  and  requirements 

27  people 


Quality  Assurance 


Suopoa  production  systems,  test¬ 
ing,  and  updating 

6  people 


Software  Systems 
ntegraoon  Engineering 


ntegration  engineering  and 
deployment  expertise 

'  6  oeooie 


Note:  :  rc'uces  Ousmess  management  and  administrative  support  personnel 


Architectures  and 
Advanced  Technology 

Appropriate  use  of  expert  systems 
and  traditional  software  through¬ 
out  CSDG 

a  people 


FIG.  2-3.  digital's  CONFIGURATION  SYSTEMS  DEVELOPMENT  GROUP 


DEC  stresses  that  total  corporate  commitment  to  AI  is  essential  and  that 
individuals  at  all  levels  throughout  the  company  must  be  educated  and  involved  if 
the  program  is  to  be  successful.  Table  2-1  lists  the  key  roles  and  functions  that  DEC 
believes  various  individuals  should  play  in  ensuring  a  successful  program.  The  table 
shows  the  importance  of  providing  corporate  direction  and  establishing  policies  that 
foster  a  supportive,  enabling  environment  and  permit  resources  to  be  committed. 
The  table  also  highlights  the  important  role  played  by  experts  and  systems  users. 
The  domain  experts  are  involved  in  systems  development  from  the  beginning  of  the 
projects.  They  and  functional  users  are  educated  in  the  processes  of  knowledge 
engineering  and  knowledge  acquisition,  thereby  enabling  them  to  more  readily  relate 


the  functional  requirements  of  the  systems  to  the  systems  developers.  Further,  the 
systems  developers  gain  a  much  better  understanding  of  the  basics  of  the  business. 


TABLE  2-1 

KEY  ROLE  MODELS  AT  DEC 


Role 

Function 

Champion 

Has  strategic  vision,  believes  in  the  technology. 

Has  political  savvy  and  ability  to  provide  supportive/enabling 
environment. 

Sponsor 

Ownership  of  business  problem,  commitment  to  solve  problem, 
organizational  stature  and  support,  appropriate  resources. 

Program  manager 

Manages  the  interface  between  players. 

Technical  team: 
knowledge  engineer 

Develops  and  tests  AI  part  of  the  system. 

Software  systems 
integration  engineer 

Designs  and  develops  traditional  part  of  system,  performs  release 
management,  and  supports  installed  systems 

Experts 

Provides  domain  knowledge. 

Users 

Provides  knowledge  of  how  system  will  fit  into  the  current/future 
business  process  and  relevant  job  satisfaction  issues. 

DEC,  however,  did  not  always  have  the  "integration”  perspective  of  AI  that 
exists  within  the  company  today.  Its  AI  program  has  evolved  through  the  following 
three  phases: 

•  Phase  I  —  Getting  Started 

•  Phase  n  —  Developing  Critical  Mass 

•  Phase  HI  —  Integration. 

For  each  of  these  three  phases.  Tables  2-2,  2-3,  and  2-4,  respectively,  show  what 
DEC  considers  specific  objectives,  unique  business/strategic  perspectives,  technology, 
and  human  resources. 

As  dec’s  AI  program  evolved  from  Phase  I  through  Phase  II  and  into  Phase  LH, 
the  AI  program  overall  was  given  different  emphasis  within  the  company.  Figure  2-4 
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TABLE  2-2 

DEC'S  PHASE  I;  GETTING  STARTED 


Objectives 

Learn  about  technology;  assess  applicability. 

Gain  management  support. 

Start  small/think  big. 

Business/strategic 

perspectives 

Start  with  important  business  problems:  create  a  corporate  vision. 

Technology 

Select  problems  big  enough  to  be  meaningful. 

Select  problems  small  enough  to  be  manageable. 

Stay  within  proven  technology  limits. 

Human  resource/ 
organization 

Select  and  train  knowledge  engineers. 

Manage  expectations. 

Assess  and  plan  for  role  of  management  information  systems. 

TABLE  2-3 

DEC'S  PHASE  II:  DEVELOPING  CRITICAL  MASS 


Objectives 

Make  expert  systems  a  reality  by  building  expert  system's  depth 
(resources,  applications,  and  commitment). 

Business/strategic 

perspectives 

Support  multiple  business  organizations. 

Provide  strategic/competitive  advantage. 

Technology 

Apply  technology  broadly: 

•  Domains 

•  Techniques/tools 

•  Paradigms 

•  Integration  with  conventional  systems. 

Human  resource/ 
organization 

Co-residency  of  knowledge  engineers  and  information  systems 
staff. 

Justify  investment  in  the  technology. 

depicts  this  transition  of  DEC’S  AI  program  along  the  difTerentiation/integration 
continuum.  During  Phase  I  when  DEC  was  just  getting  started  in  AJ,  management 
believed  that  difTerentiating  AI  technologies  from  conventional  methods  and  solution 
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TABLE  2-4 


DEC'S  PHASE  III:  INTEGRATION 


Objectives 

Build  a  "knowledge  network." 

Use  the  technology  to  integrate  the  business  functions. 

Business/strategic 

Achieve 

perspectives 

•  Productivity 

•  Predictability 

•  Control. 

Technology 

Appropriate  application  mix  of  expert  systems  and  conventional 
technologies. 

Right  method  for  the  right  problem. 

Human  resource/ 
organization 

Business  and  technology  planning  done  in  concert. 

techniques  was  important.  Management  gave  the  program  considerable  special 
attention  and  stressed  its  awareness  and  support  of  the  technology.  During  Phase  I, 
DEC  emphasized  the  achievement  of  small,  real-world  business  successes  using  AJ. 
As  DEC  moved  into  Phase  n  and  began  to  build  critical  mass,  management  relaxed 
emphasis  on  differentiation  of  the  program.  It  began  to  actively  examine  ways  to 
integrate  AJ  methodologies  with  more  conventional  systems  and  techniques.  In 
Phase  n,  DEC  management  emphasized  the  business  organizations  and  the  ability  to 
provide  them  with  strategic,  competitive  advantage;  it  stressed  the  business 
perspective  since  the  technology  perspective  had  been  given  more  emphasis  during 
Phase  1.  Finally,  during  Phase  Eli  management  placed  emphasis  on  applying  AI 
along  with  other  technologies  to  ''reate  an  integrated  knowledge  network  throughout 
the  company.  By  integrating  AJ  with  existing  systems  and  methods,  the  company  is 
able  to  extract  a  greater  return  on  investment  (ROD  by  better  leveraging  the  prior 
investment  in  the  existing  information  system  infrastructure.  Further,  the  emphasis 
during  Phase  EH  is  on  the  corporate  processes  rather  than  on  the  separate  functions 
that  comprise  the  processes. 

International  Business  Machines  Corporation 

In  1985,  IBM  made  a  top-level  management  commitment  to  use  AI.  Its 
Corporate  Management  Committee,  comprised  of  the  CEO,  Chairman  of  the  Board, 
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FIG.  2-4.  DEC'S  DIFFERENTIATION/INTEGRATION  SPECTRUM 


and  several  Vice  Presidents,  tasked  the  Vice  President  of  Systems  Research  Division 
to  head  the  effort  (see  Figure  2-5).  As  shown,  IBM,  like  DEC,  has  a  very  senior  level 
group  tasked  with  providing  policy  guidance  and  direction  for  the  AI  program  within 
IBM.  This  group  is  IBM’s  AI  Steering  Committee. 


Note:  AlPO  =  AI  Pro)ea  Offico;  AISC  =  AI  Support  Center;  K8S  =  knowledge-based  systems 

FIG.  2-5.  AI  STRUCTURE  IN  IBM 


In  1985,  IBM  established  an  AI  Project  Office  (AIPO)  as  a  temporary 
organization  that  was  to  exist  for  approximately  3  years;  it  was  officially  disbanded 
on  30  June  1989.  IBM  established  the  AIPO  to  solidify  its  project  plan  for  expert 
systems,  to  develop  an  internal  AI  use  program,  and  to  develop  marketing  strategies 
and  plans  for  external  sales  of  such  IBM  AI  technology  products  as  its  expert  systems 


development  software.  Fourteen  persons  manned  the  project  office  and  worked  to 
help  inform  management  of  the  AI  program,  to  provide  management  with  an 
understanding  of  AI  initiatives  already  under  way  within  IBM,  and  to  integrate  the 
varied  and  dispersed  expertise  and  tools  already  available  within  IBM.  The  project 
office  also  provided  seed  money  to  fund  strategic  AI  applications  in  IBM. 

IBM  also  established  an  AI  Support  Center  ( AISC)  to  provide  support  to  organic 
development  efforts.  It  serves  as  a  central  organization  to  provide  consulting 
services,  education  and  tx  aining  services,  information  services,  and  ouiiuixuiiications 
services  throughout  IBM.  Table  2-5  presents  the  functions  assigned  to  IBM’s  AISC. 
Unlike  the  Project  Management  Office  (PMO),  the  AISC  has  a  permanent  mandate. 
The  AISC  trains  approximately  1,000  IBM  personnel  each  year  and  helps  61  different 
user  groups  by  sponsoring  conferences  and  by  collecting  and  distributing  technical 
information  and  lessons  learned. 


TABLE  2-5 

FUNCTIONS  OF  IBM's  AI  SUPPORT  CENTER 


Role 

Function 

Education 

Introductory/overview  basic  skills 

Consulting 

Opportunity  identification 

Technology  selection 

Prototyping 

Problem  assistance 

Clearing  house 

Application  inventory 

Communications 

User  groups 

Conferences 

Electronic  note  boards 

Knowledge-based  systems  (KBS)  development  activity  within  IBM  is  both 
centralized  and  decentralized.  Nearly  800  IBM  personnel  are  actively  involved  in 
decentralized  systems  develcpinent.  However,  IBM  also  has  a  central  group  of 
developers  located  at  their  KBS  Development  Laboratory  (see  Figure  2-5).  IBM,  like 
DEC,  has  established  key  roles  for  individuals  at  all  management  levels.  Those  roles 


are  shown  in  Table  2-6.  Senior  management  determines  how  expert  systems  fit  into 
the  company’s  overall  strategy.  Line  managers  identify  projects  and  secure  expert 
involvement  and  funding.  Domain  experts  are  responsible  for  transferring  functional 
and  process  knowledge  to  the  systems  developers. 

TABLE  2-6 

MANAGEMENT  ROLES  AT  IBM 


Position 

Function 

Senior  Executives 

Strategy 

Organization 

Line  Executives 

Project  direction 

Domain  experts 

Funding 

Domain  Experts 

Knowledge  transfer 

Marketing 

Du  Pont  Company 

Du  Pont  began  its  involvement  in  AI  in  1985  with  efforts  to  develop  small 
in-house  expert  systems.  Unlike  DEC,  which  placed  primary  emphasis  on  the 
development  of  large  integrated  systems,  Du  Pont  was  interested  in  having  users 
build  and  maintain  their  own  systems  to  address  narrow  problems  within  specific 
functional  areas.  DuPont  established  an  objective  of  making  every  individual 
throughout  the  company  aware  of  AI  technology. 

Dr.  Edward  Mahler,  the  Director  of  Du  Pont’s  AI  program,  wanted  to  train 
every  engineer  within  the  company  to  be  as  familiar  with  KBS  development  tools  as 
they  were  with  Lotus  1-2-3  spreadsheets.  Development  by  actual  users  on  their  PCs 
was  the  order  of  the  day.  Because  no  commercially  available  expert  system 
development  tool  was  deemed  simple  enough  to  be  used  by  non-AI  specialists  in  1985, 
Du  Pont  developed  its  own  user-friendly  expert  system  development  tool  called  "Tool 
Kit.”  Today,  more  than  700  copies  of  Tool  Kit  have  been  distributed  to  Du  Pont’s  120 
plants  throughout  the  world.  In  addition,  Du  Pont  has  since  purchased  a  company- 
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wide  license  to  a  recent,  commercially  available  development  tool.  More  than  700 
copies  of  this  newer  tool  have  also  been  distributed  throughout  Du  Pont. 

Dr.  Mahler  served  as  the  early  "champion”  of  AI  within  Du  Pont,  using  his  own 
office  automation  budget  as  seed  money  to  start  initiatives.  He  also  used  his  "old  boy 
network”  to  attract  volunteer  staff  to  help  him  get  the  activities  under  way.  These 
early  efforts  attracted  the  attention  of  his  supervisor,  the  Vice  President  for 
Information  Systems,  who.  in  turn,  reports  to  the  Senior  Vice  President  for 
Technology.  Senior  management  within  Du  Pont  then  decided  to  consolidate 
Dr.  Mahler’s  activities  and  formally  established  a  small  AI  Program  Office.  A  small 
support  group  of  12  persons  was  tasked  to  "catalyze  the  application  of  artificial 
intelligence  techniques,  particularly  expert  systems,  broadly  and  effectively  through 
Du  Pont”  [3].  Dr.  Mahler  was  named  Du  Pont’s  Program  Manager  for  AI.  Six  of  his 
12  staff  members  are  assigned  on  a  permanent  basis  to  the  program  office.  The  other 
six  individuals  are  assigned  on  a  temporary  basis  from  other  parts  of  Du  Pont. 

The  functions  of  Du  Pont’s  AI  program  office  include  teaching,  consulting,  and 
operating  a  telephone  help  line.  The  central  group  also  acquires  corporate  licenses  to 
commercial  shells  to  lower  the  costs  of  systems  development  for  Du  Pont’s  various 
users.  Training  provided  by  the  group  includes  a  management  awareness  briefing,  a 
4-hour  course  on  expert  systems  for  managers,  and  a  2-day  training  course  during 
which  functional  experts  learn  to  use  the  shells  to  develop  and  maintain  their  own 
systems.  The  central  group  also  assesses  new  expert  systems  development  tools  and 
offers  advice  to  individuals  throughout  Du  Pont  who  are  considering  their  purchase. 

Recently,  Du  Pont  has  also  begun  the  development  of  much  larger  systems  that 
integrate  knowledge  from  various  functional  users  and  combine  expert  systems 
techniques  with  conventional  programs  and  operations  research  models.  These 
larger,  more  complex  systems  are  being  developed  by  AI  specialists  under  contract  to 
Du  Pont  as  well  as  by  organic  teams  of  functional  experts  and  traditional  manage¬ 
ment  information  system  staffs. 

Lockheed  Corporation 

Lockheed’s  organized  involvement  with  AI  stems  from  1985  when  a  corporate 
task  force  decided  that  the  company  needed  to  understand  and  use  the  technology  and 
recommended  that  it  establish  the  Artificial  Intelligence  Center  (AIC).  Lockheed 
companies,  however,  had  been  conducting  AI  research  since  1981.  As  shown  in 
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Figure  2-6,  the  AIC  is  a  part  of  Lockheed’s  Missiles  and  Space  Systems  Group  and 
reports  to  the  Research  and  Development  Division  of  that  group.  Figure  2-7  depicts 
the  organizational  structure  of  the  AIC  and  clearly  shows  the  center’s  three  major 
functions  of  technology  development,  technology  transfer,  and  training  and 
education.  The  AIC  has  a  permanent  staff  of  approximately  50  researchers  and 
trainers  and  houses  visiting  staff  members  from  other  Lockheed  companies  and 
divisions.  The  AIC  also  employs  part-time  non-Lockheed  staff,  including  professors, 
graduate  students,  and  contracted  assistants  [18]. 


FIG.  2-6.  Al  WITHIN  LOCKHEED'S  CORPORATE  STRUCTURE 


The  roles  of  the  AIC  are  quite  similar  to  those  of  the  central  AI  support 
activities  of  all  the  other  companies  reviewed  herein.  Technology  transfer  including 
the  assimilation  of  current  information  on  company-wide  AI  activities  and 
applications,  technical  consulting,  and  the  identification  of  key  business  require¬ 
ments  or  problems  to  which  the  technology  can  be  advantageously  applied  are 
necessary  activities  for  all  of  the  companies’  successful  use  of  the  AI  technologies. 
Training  and  education  are  also  necessary  and  important  activities. 
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•  Development,  verifica¬ 
tion,  and  support  of  Al 
tools 

•  Industry  contracts 

•  University  research 

•  Applied  research 


•  Identification  and  res¬ 
olution  of  Lockheed 
technology  needs 

•  Technical  consulting 

•  Al  database 


•  Residency  program 

•  Special  technical  man¬ 
agement  courses 

•  Continuing  education 
courses  and  seminars 


FIG.  2-7.  LOCKHEED  AlC  ORGANIZATION 


Lockheed  also  plays  an  active  role  in  technology  development.  It  maintains 
close  ties  with  university  Al  researchers.  In  addition,  other  contractors’  services  are 
used,  when  appropriate.  Lockheed  also  invests  heavily  in  applied  Al  research, 
including  tool  development. 

Lockheed  has  a  comprehensive  corporate  Al  education  and  training  program.  It 
includes  a  management  education  program,  a  continuing  education  program,  and  a 
residency  program.  The  6-month  residency  program  is  depicted  in  Figure  2-8. 

Ford  Motor  Company 

Ford  formally  got  involved  in  Al  in  the  early  1980s,  realizing  that  a  number  of 
its  major  business  problems  were  amenable  to  Al.  One  of  its  first  systems  was  the 
Service  Bay  Diagnostician,  which  was  to  be  used  by  Ford’s  automotive  dealers  to 
assist  their  maintenance  personnel  with  car  repairs.  Ford  felt  a  need  for  the  system 
in  the  face  of  the  increasing  complexity  of  its  vehicles  and  the  high  turnover  of 
maintenance  personnel.  The  goal  of  the  Service  Bay  Diagnostic  System  was  to  use 
the  expert  system  to  guide  the  mechanic  through  the  vehicle  repair  process.  The 
expert  system  used  data  collected  from  the  vehicle’s  on-board  computers  and  also 
queried  the  mechanic  for  input. 

In  1984,  Ford  Motor  Company  purchased  equity  in  both  Inference  Corporation 
and  the  Carnegie  Group,  two  leading  Al  firms.  Also  as  early  as  1984,  Ford  had  an 
unofficial  grassroots  group,  the  Ford  Al  Working  Group,  serving  the  needs  of  Al 
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FIG.  2-8.  LOCKHEED'S  AlC  RESIDENCY  PROGRAM 


practitioners.  The  Ford  AI  Working  Group  had  no  formal  support  but  arose  from  the 
need  of  small  groups  within  the  operating  divisions  to  share  information  and 
expertise  in  AJ. 

Ford  is  firmly  committed  to  the  use  of  rule-based  expert  systems  for  the 
diagnostics  of  robotic  systems  used  in  its  manufacturing  operations.  The  robotics 
group  at  Ford  Motor  Company  funded  a  pilot  project  to  diagnose  robot  malfunctions, 
and  the  project  proved  quite  successful.  Ford  then  notified  its  robotics  suppliers  that 
Ford  would  only  purchase  robotic  systems  from  vendors  who  offered  accompanying 
diagnostic  expert  systems.  For  those  robotic  manufacturers  who  did  not  have  expert 
systems  capabilities  in-house,  Ford  recommended  companies  that  can  develop  the 
expert  systems. 

The  funding  of  the  robot  diagnostic  expert  system  pilot  is  being  used  as  a 
technology  transition  model  within  Ford.  A  similar  project  is  now  under  way  at  Ford 
Aerospace’s  Sunnjrvale,  California,  division  to  develop  a  diagnostic  expert  system  for 
a  laser-guided  welder.  Ford  is  again  funding  the  development  of  the  pilot  system. 
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Once  successful,  the  pilot  system  will  determine  the  specifications  for  the  operational 
expert  system  to  be  delivered  by  the  manufacturers.  The  manufacturer  of  the 
laser-guided  welding  system  must  develop  the  ability  to  deliver  future  expert 
systems  that  meet  these  specifications  [19]. 

Expert  Systems  have  also  been  used  at  Ford  for  functions  that  range  from  credit 
approval  to  product  design.  These  applications  have  been  developed  both  under 
contract  and  organically.  To  support  their  organic  development,  Ford  Aerospace 
developed  a  model-based  reasoning  tool  called  Paragon. 

In  early  1986,  Dr.  Tony  Tether,  Vice  President  of  Technology  and  Advanced 
Development  for  Ford  Aerospace,  directed  the  formation  of  an  A1  consortium  to  be 
composed  of  senior  members  of  Ford  Aerospace’s  operating  entities.  The  purpose  of 
the  AI  consortium  was  to  elevate  technology  evaluation  within  Ford  Aerospace.  The 
consortium  was  tasked  to  formulate  a  business  planning  document  for  the  application 
and  cooperative  use  and  development  of  AI  technologies  within  Ford.  The  initiative 
received  high-level  support  from  Mr.  John  Betti,  Ford  Motor  Company’s  former  Vice 
President  of  Diversified  Products  Operations  and  now  Under  Secretary  of  Defense  for 
Acquisition. 

Today,  the  AI  consortium  continues  to  meet  and  is  responsible  for  AI  business 
planning  and  research  coordination.  The  AI  Working  Group  continues  to  provide 
communication  and  technology  transfer  functions.  However,  all  technology 
development  is  done  in  support  of  the  business  plan.  AI  research  is  funded  by 
Government  contracts  and  applications  development  is  funded  by  Ford’s  internal 
clients  [20]. 

General  Motors 

GM’s  participation  in  AI  technologies  predates  its  current  embodiment  within 
its  Advanced  Engineering  Staff  (AES).  General  Motors  Research  (GMR)  Labor¬ 
atories  started  robotics  work  in  the  early  1970s,  and  by  the  late  1970s  GM’s 
Manufacturing  Staff,  part  of  Manufacturing,  Engineering  and  Development,  was 
using  robotics.  The  first  corporate-wide  robotics  symposium  with  talks,  exhibits,  and 
application  demonstrations  occurred  around  1980.  GMR  also  initiated  GM’s  effort  in 
machine  vision/perception  in  the  late  1960s  and  early  1970s.  The  effort  was  less 
"mature”  than  the  level  of  robotics  occurring  at  the  same  time.  By  the  mid-1970s, 
Delco  Electronics  was  using  a  machine  vision  application  to  look  at  integrated-circuit 
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chip  orientation.  Other  machine  vision  applications  were  developed  for  operating 
divisions  jointly  with  GMR  [21], 

Around  1985  —  86  AES  started  a  natural  language  processing  group.  Tools  were 
available,  but  the  vendors  were  not  ready  to  develop  applications  in  order  to  leverage 
the  technology.  In  addition,  the  Advanced  Manufacturing  Staff  and  then  later  the  AJ 
group  did  some  development  in  voice  synthesis  and  voice  processing.  In  1985  an 
expert  systems  group  was  added  to  AES’s  Machine  Intelligence  Technology 
Implementation  group  and  the  name  was  changed  to  the  Artificial  Intelligence  (AI) 
group.  The  Machine  Intelligence  group  and  the  AI  group  were  combined  at  about  the 
same  time  in  late  1985  or  early  1986.  In  1986  restructuring  and  the  addition  of 
natural  language  technologies  and  robotics  to  AI  gave  rise  to  the  current  structure  of 
three  groups  within  the  AI  group;  Vision,  Robotics,  and  Cognitive  Systems 
(containing  knowledge-based  and  expert  systems  and  natural  language 
technologies). 

As  depicted  in  Figure  2-9,  the  CM  AI  group  is  a  part  of  AES,  which  in  turn 
reports  to  the  Technical  Staffs  Group  (TSG)  and  its  group  Vice  President.  The  TSG 
oversees  Patents,  GMR,  the  Design  Staff,  the  Current  Engineering  and  Manu¬ 
facturing  Services  Staff  (CEMSS),  and  AES.  TSG’s  group  Vice  President  reports  to 
an  Executive  Vice  President  (or  a  Vice  Chairman)  who  also  oversee  Electronic  Data 
Systems  (EDS)  and  Hughes  in  addition  to  TSG.  The  Executive  Vice  President  reports 
directly  to  the  CEO. 

The  Corporate  Robotics  and  Machine  Intelligence  Council,  an  adjunct  to  the  AI 
group,  is  made  up  of  divisional  representatives  and  meets  monthly  to  solve  problems 
and  share  information  and  ideas. 

The  AI  group  at  GM  does  a  "technology  push.”  Divisions  decide  what  they  will 
incorporate.  Usually,  additions  are  small,  point  implementations.  Occasionally,  for 
example,  with  the  addition  of  a  new  plant,  bigger  implementations  are  planned.  AES 
works  to  transfer  technology,  not  direct  technology. 

AES  has  three  objectives  in  technology  transfer:  to  discover  and  develop  new 
technology,  to  deploy  that  new  technology,  and  to  transfer  new  technology  to  the 
divisions.  Technology  transfer  proposes  to  impart  an  understanding  for  the  purpose 
of  proliferation  and  maintenance  throughout  GM.  For  example,  Packard  Electric 
was  the  first  to  use  machine  vision  from  the  AI  group;  since  then,  it  has  installed 
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more  than  100  machine  vision  systems.  AES  may  provide  seed  money,  but  the 
division  has  to  pick  up  the  application  after  that. 

GM’s  corporate  environment  is  an  order  of  magnitude  greater  than,  say,  DEC’s. 
A  GM  division  alone  may  be  equal  in  size  to  all  of  DEC.  Therefore,  only  a  few 
applications  have  the  broad  applicability  that  warrants  central  support.  Spot¬ 
welding  is  one  positive  example  with  broad  applicability. 

AES  tracks  and  communicates  lessons  learned.  Every  installation  is  recorded 
in  an  applications  database.  Communication  occurs  in  the  monthly  Corporate 
Robotics  and  Machine  Intelligence  Council  (CRMIC)  meetings  and  the  annual 
Manufacturing  Technology  Conference.  AI  is  also  frequently  featured  in  GM’s 
internal  publications. 

CURRENT  DoO  AI  LOGISTICS  INITIATIVES 

Within  the  production  and  logistics  disciplines,  DoD  Components  have  been 
working,  for  the  most  part  separately,  to  apply  AI  techniques.  The  Army  is 
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developing  AI  applications  at  several  AI  applications  centers.  One  such  Army 
Logistics  AI  Applications  Center  is  located  at  Fort  Lee,  Virginia;  additional  logistics 
applications  are  being  developed  by  the  Army  AI  Center  in  The  Pentagon,  by  the 
Army  AI  Training  Center  at  Fort  Gordon,  Georgia,  and  by  the  Materiel  Readiness 
Support  Activity. 

The  Air  Force  has  established  an  AI  Prograun  Management  Office  at  the  AFLC 
Headquarters  at  Wright  Patterson  Air  Force  Base  (AFB),  Ohio.  The  office  currently 
has  more  than  20  staff  members.  The  AFLC  AI  PMO  is  organized  as  shown  in 
Figures  2-10  and  2-11.  A  1987  study  of  lessons  learned  by  Fortune  500  companies  in 
developing  and  fielding  AI  systems  [22]  served  as  the  basis  for  the  AFLC  AI  PMO 
organizational  structure  and  functions.  Figures  2-10  and  2-11  depict  the  range  of 
services  provided  by  this  central  organization.  Note  that  many  of  the  functions  of  the 
Planning,  Procedures,  and  Applications  Branch  involve  policy  determination  and 
guidance. 


•  Concept  utilization 

•  Qualification 

•  Acauisitions 

•  Integration 


•  Organic  execution 
(otner  than  ISF  and 
ATE  efforts) 


•  Wessons  learned 

•  Productivity  analysis 

•  Budget  preparation 

•  Program  advocacy 


Vote;  ATE  =  Automatic  Test  Equipment;  iSF  =  integrated  Support  Facility 


FIG.  2-10.  AFLC  AI  PROGRAM  MANAGEMENT  OFFICE 


The  AFLC  AI  PMO  was  to  be  a  temporary  management  organization  tasked 
with  ensuring  awareness  of  AI  throughout  AFLC  and  providing  for  the  successful 
transition  of  the  technology  via  successful  fielded  applications.  The  AFLC  AI 
Applications  Support  Center  (AIASC),  however,  was  given  a  permanent  mandate. 
The  two  organizations,  AI  PMO  and  AIASC,  were  purposely  collocated  to  ke  p  scarce 
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FIG.  2-11.  AFLC  Al  APPLICATIONS  SUPPORT  CENTER 


AI  talent  together.  As  depicted  in  Figure  2-11,  the  AIASC  was  responsible  for 
technical  development,  evaluation,  and  support  activities. 

Development  activity  in  AFLC  is  dispersed  throughout  the  Command.  AI 
systems  development  occurs  at  subordinate  field  activities  and  also  within  the 
AIASC.  In  addition,  AFLC  contracts  for  the  development  of  large  complex  AI 
systems  or  systems  that  must  be  integrated  with  existing  information  and  decision 
support  systems.  Numerous  other  centers  of  excellence  in  AI  throughout  the  Air 
Force  are  also  exploring  logistics  AI  applications.  Those  centers  include  the  Human 
Resources  Laboratory,  the  Air  Force  Institute  of  Technology,  the  Rome  Air 
Development  Center,  the  Strategic  Air  Command,  the  Military  Airlift  Command, 
and  the  Air  Force’s  Small  Computer  Systems  Center  at  Gunter  AFB,  Alabama. 

The  Navy  is  also  pursuing  logistics  AI  initiatives  and  also  has  numerous 
centers  of  excellence  including  the  Navy  AI  Laboratory  in  Washington,  D.C.,  the 
Naval  Avionics  Center,  and  the  Navy  Supply  Systems  Command.  The  Navy  has 
pursued  the  development  of  several  systems  through  the  Small  Business  Innovative 
Research  Program. 

The  Defense  Logistics  Agency  (DLA)  has  established  a  small  central  group  of 
four  individuals  at  the  DLA  Systems  Automation  Center  (DSAC).  That  group 
supports  others  throughout  DLA  who  are  attempting  to  apply  AI  technologies.  DLA 
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has  also  started  an  active  DLA  AI  Working  Group  and  begun  management 
awareness  training.  Some  training  is  conducted  organically,  while  other  training 
services  have  been  contracted.  The  Marine  Corps  and  OSD  staff  activities  are  also 
actively  investigating  AI  logistics  applications. 

Attempt’'*  to  coordinate  the  Services’  activities  have  begun.  In  138G,  the 
Logistics  AI  Coordinating  Cell  (LAICC)  was  chartered  by  the  Conference  of  the 
Logistics  Directors  (COLD).  The  LAICC  was  tasked  to  foster  open  communications 
between  the  Services  and  DoD  agencies  on  AI  logistics  projects.  The  LAICC  is 
chaired  by  a  representative  of  the  Office  of  the  Assistant  Secretary  of  Defense 
(Production  and  Logistics)  [OASD(P&L)]  and  includes  representatives  from  the 
Director  for  Studies,  Concepts  and  Analysis,  JCS  (J4);  and  the  Offices  of  the  Deputy 
Chief  of  Staff  for  Logistics,  U.S.  Army;  the  Deputy  Chief  of  Naval  Operations 
(Logistics);  the  Deputy  Chief  of  Staff  Logistics  and  Engineering,  Air  Force;  the 
Deputy  Chief  of  Staff  Installation  and  Logistics,  Marine  Corps;  the  Office  of  the 
Director,  Defense  Logistics  Agency,  and  the  Director,  Strategic  Defense  Initiative 
Organization. 

Each  year  since  1987,  the  LAICC,  together  with  the  J4  Logistics  Directorate 
and  the  American  Defense  Preparedness  Association,  has  sponsored  an  annual 
Symposium  and  Workshop  on  Artificial  Intelligence  Applications  for  Military 
Logistics  at  Williamsburg,  Virginia.  The  conference  allows  the  exchange  of 
information  among  the  Services  on  their  AI  logistics  activities. 

In  addition,  P&L  and  the  LAICC  have  actively  pursued  Productivity  Enhancing 
Capital  Investment  (PECI)  funding  for  logistics  AI  initiatives.  In  May  1988,  the  AI 
production  and  logistics  applications  program  was  certified  as  a  priority  candidate  for 
PECI  funds.  The  LAICC  worked  with  the  Services  to  rank  order  projects  submitted 
for  funding.  Fifty-three  FY90  projects  valued  at  $26.3  million  were  approved  for  all 
of  the  Services.  Ten  FY91  projects  valued  at  $5.3  million  have  also  been  approved. 

SUMMARY  OF  INDUSTRY  AND  DoD  AI  INITIATIVES  AND  PHILOSOPHIES 

The  preceding  review  of  companies,  Services,  and  DoD  agencies  reveals  a 
number  of  similarities  among  the  functions  or  activities  required  to  support  the 
organization’s  involvement  with  the  AI  technologies.  Recurrent  themes  include  a 
requirement  to  pursue  AI  as  a  technique  capable  of  addressing  key  business  needs. 
The  logistics  needs,  not  the  technologies,  provide  the  focus.  Because  the  focus  is  on 
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the  business  requirement  and  because  the  technologies  are  viewed  as  providing  key 
strategic  and  competitive  advantages,  senior  corporate  involvement  is  viewed  as 
essential  in  providing  strategic  direction  and  resources  in  the  use  of  the  technologies 
as  well  as  in  providing  policy  guidance.  Several  companies  formalize  this  activity  in 
groups  reporting  directly  to  the  company's  senior  officers.  AI  Boards  of  Directors  and 
AI  Steering  Groups  are  common  within  industry. 

Another  recurrent  theme  is  the  important  role  played  by  the  domain  experts  in 
the  development  and  maintenance  of  these  systems.  Further,  specific  roles  are 
performed  by  all  levels  of  management  throughout  the  company  or  organization, 
necessitating  that  all  be  aware  of  the  technologies  and  their  capabilities  and 
limitations.  Training  at  all  levels  of  management  is  suggested;  however,  emphasis  is 
placed  on  providing  knowledge  engineering  and  knowledge  acquisition  skills  to 
domain  practitioners  to  permit  them  to  better  relate  functional  information  and 
knowledge  system  requirements  to  systems  development  personnel. 

All  the  companies  and  organizations  examined  pursued  both  centralized  and 
decentralized  activities  and,  both  large  and  small,  simple  and  complex  AI  systems 
development  initiatives.  Figure  2-12  provides  a  framework  for  examining  and 
appropriately  placing  an  organization’s  AI  application  activities.  The  discussion  of 
both  dec’s  and  Du  Font’s  AI  activities  demonstrated  how  companies  "differentiate” 
the  technology  by  working  on  small  systems  that  demonstrate  the  feasibility  of  the 
technology  early  on.  As  the  company  gains  an  understanding  and  acceptance  of  AI 
technologies  over  time  and  builds  "critical  mass,”  the  philosophy  of  "differentiation” 
gives  way  to  a  philosophy  of  "integration.”  Here,  larger,  more  sophisticated  systems 
tend  to  be  built  which  combine  knowledge  from  multi-functional  areas  and  integrate 
AI  technologies  with  other  appropriate  conventional  systems  and  decision  support 
techniques. 

Another  recurrent  theme  is  the  need  for  a  central  group  to  provide  technical 
consulting  and  information  services.  The  most  common  form  of  information  service  is 
an  applications  database.  Technical  consulting  services  include  problem  selection 
consulting  and  education  and  training  services. 
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FIG.  2-12.  APPLICATION  ACTIVITY 
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CHAPTERS 


A  DoD  LOGISTICS  ARTIFICIAL  INTELLIGENCE  PROGRAM 

THE  NEED  FOR  AN  Al  LOGISTICS  PROGRAM 

The  ASD(P&L)  should  move  quickly  to  take  advantage  of  AI  applications  that 
the  private  sector  has  demonstrated  to  generate  logistics  productivity  and  quality 
improvements  in  the  range  of  5  to  25  percent.  DoD  is  already  operating  with 
direction  from  the  White  House  to  achieve  a  3  percent  annual  productivity  increase. 
On  27  April  1988,  Executive  Order  12637  set  a  Government-wide  3  percent 
productivity  improvement  goal  for  appropriate  functions.  The  DoD  FY90 
Productivity  Improvement  Plan  is  currently  the  major  tool  for  achieving  that  goal. 
The  program  contains  63  AI  initiatives  in  DoD  functions  and  programs  valued  at 
approximately  $31.6  million  over  FY90  and  FY91.  Furthermore,  DoD  is  imple¬ 
menting  the  philosophy  of  Total  Quality  Management  (TQM)  as  a  prime  means  to 
increase  productivity  through  continuous,  incremental  improvement  in  quality. 

The  objective  of  DoD  management  is  to  make  the  best  use  of  national 
defense  resources  to  provide  the  means  to  preserve  our  national  security  in 
peace  and  in  war.  This  is  a  challenge  of  significant  magnitude.  The 
Department  employs  one  million  civilians  and  two  million  active  duty 
military  personnel,  manages  1,200  installations  spread  around  the  globe, 
and  for  FY90  is  submitting  a  budget  that  would  give  it  the  authority  to 
spend  $293.8  billion.  DoD  executes  more  than  15  million  contract  actions 
each  year.  Last  year,  (FY89),  the  total  value  of  those  contracts  reached 
$160  billion  [23|. 

The  commercial  sector  is  actively  pursuing  logistics  applications  of  AI  and  has 
established  management  structures  to  facilitate  these  applications,  as  discussed  in 
Chapter  2.  In  addition,  a  recently  published  report  about  trends  in  highly 
progressive  and  successful  commercial  logistics  firms  indicates  that  those  firms  are 
actively  pursuing  AI  and  knowledge-based  systems.  More  than  one-fifth  of  the 
manufacturers  responding  indicated  plans  to  install  logistics  AI  applications. 
Leading  edge  wholesalers  and  retailers  also  had  plans  to  apply  AI  technologies  to 
logistics  [24], 

Although  AI  technologies  provide  significant  increases  in  weapon  systems’ 
operational  capabilities,  the  technologies  offer  their  greatest  potential  in  terms  of 
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improving  the  effectiveness,  efficiency,  and  quality  of  DoD’s  logistics  operations  and 
management.  These  systems  contain  DoD’s  most  valuable  resource  —  the  knowledge 
of  how  to  successfully  plan,  budget,  and  manage  the  support  of  our  forces.  Because 
they  enable  us  to  capture,  enhance,  and  reuse  our  knowledge  of  DoD  logistics,  they 
represent  a  strategic  technology  that  should  be  properly  leveraged  and  controlled. 
For  example,  these  systems  can  ensure  consistent  logistics  policy  is  implemented  by 
all  branches  of  the  Armed  Forces.  Naturally,  we  must  ensure  the  policy,  i.e., 
knowledge,  contained  in  these  systems,  is  correct  and  timely. 

We  recommend  that  the  DoD  establish  a  formal  program  to  accelerate  the 
implementation  of  AI  technologies  within  production  and  logistics  activities.  The 
program  should  encompass  the  following  DoD  functions;  procurement, 
manufacturing,  logistics,  and  environment  and  facilities  management.  The  DoD 
Components  affected  should  be  OSD,  the  Military  Departments,  Defense  agencies, 
and  the  JCS. 

OBJECTIVES  OF  THE  AI  PROGRAM  FOR  DoD  PRODUCTION 
AND  LOGISTICS  ACTIVITIES 

The  objectives  of  the  P&L  AI  program  and  logistics-related  activities  should  be 
to  promote  the  innovative  use  of  AI  technologies  in  DoD  logistics;  to  improve  the 
quality,  timeliness,  and  uniformity  of  logistics  decision  making;  to  increase 
productivity  in  the  work  force;  and  to  reduce  the  cost  of  ownership  of  DoD  systems. 
The  Military  Departments,  DLA,  and  their  field  activities  should  retain  program 
management  control  over  all  AI  projects.  However,  they  should  be  obligated  to  share 
program  information,  technology  applications,  and  their  lessons  learned  and 
assessment  of  the  results  with  all  other  DoD  elements. 

REQUIRED  SERVICES  FOR  A  SUCCESSFUL  LOGISTICS  AI  PROGRAM 

Based  upon  a  review  of  corporate  and  DoD  AI  initiatives,  we  developed  a 
tentative  list  of  the  services  required  to  enable  and  support  successful  AI 
applications.  This  list  was  further  refined  through  extensive  interviews  with  users  to 
determine  their  needs  and  desires.  Both  sophisticated  AI  developers  and  novices 
throughout  the  Military  Departments  and  Defense  agencies  were  surveyed  to 
determine  the  services  that  they  would  like  to  see  made  available  to  support  their  AI 
activities. 
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This  user  requirements  survey  confirmed  that  organizations  are  consistently 
interested  in  having  AI  informational  and  technical  services  available  to  support 
their  activities.  The  services  they  requested  are  listed  below  in  priority  order,  with 
the  highest  priority  first; 

•  Information 

•  Technical  support 

•  Technical  assessment 

•  Programming 

•  Communications 

•  Management  Support 

•  Methodology. 

Table  3-1  depicts  each  of  these  services  and  the  specific  functions  that  each 
encompasses.  Each  of  these  services  will  be  discussed  in  turn. 

Information  Services 

Our  survey  indicates  that  the  logistics  community  needs  information  services 
more  than  any  other  type  of  service.  As  shown  in  Table  3-1,  four  primary  information 
service  categories  were  identified  during  the  interview  process.  All  the  inteiwiewees 
mentioned  a  need  for  current  applications  databases  and  stressed  how  important  it  is 
that  those  databases  contain  systems’  descriptions  that  are  understandable  and  do 
not  overuse  acronyms.  Interviewees  also  frequently  requested  information  on 
current  tool  capabilities,  hardware  requirements  of  those  tools,  and  cost  information. 
Numerous  individuals  suggested  a  tools  database  that  could  be  readily  accessed. 
Individuals  also  requested  access  to  a  database  of  lessons  learned  on  past 
development  efforts.  Finally,  some  persons  requested  basic  reference  materials  such 
as  current  articles  and  technical  reports. 

The  general  consensus  is  that  a  great  deal  of  information  is  available.  In  fact, 
this  information  is  literally  increasing  exponentially  on  a  daily  basis.  The  inter¬ 
viewees  need  assistance  in  identifying,  filtering,  applying,  and  retaining  the 
available  information.  The  community  feels  that  an  organization  capable  of 
understanding  the  technical  details  and  managerial  implications  of  the  AI  literature 
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TABLE  3-1 


REQUIRED  ACTIVITIES  FOR  A  SUCCESSFUL  LOGISTICS  Al  PROGRAM 


Service 

Function 

Information 

•  Applications  database 

•  Tool  database 

•  Lessons  learned  database 

•  Source  library 

Technical  support 

•  Technical  consulting 

►  Problem  selection  and  scoping 

►  Tool  selection 

•  Al  hotline  support 

•  Training  and  education 

Technical  assessment 

•  Applications  evaluation  and  grading 

•  Tools  evaluation  and  new  tools  evaluation 

•  Technology  assessment 

Programming 

•  Systems  development 

•  Systems  enhancement  and  maintenance 

•  Systems  test  and  evaluation 

•  Configuration  management 

•  Systems  documentation 

Communications 

•  Electronic  forms 

•  Newsletters 

•  Conferences 

Management  support 

•  ROl  analyses 

•  Contracting  services 

•  Budget  preparation 

•  Tool/application  distribution 

Methodology 

•  Systems  Verification  and  Validation 

•  Knowledge  engineering 

•  Cost/benefit  analysis 

would  serve  as  a  valuable  resource.  This  function  would  be  the  same  for  all  Services 
and  DoD  agencies. 

Technical  Support  Services 

The  second  most  requested  service  is  technical  support.  In  that  area, 
consulting,  telephone  hotline,  and  training  are  the  primary  categories  of  interest. 
Many  individuals  believe  that  technical  support  services  are  difficult  to  access.  In 
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many  cases,  the  requirements  are  not  sufficient  to  warrant  contracting  for  the 
services,  but  few  alternatives  are  available.  Thousands  of  users  request  technical 
support  and  only  a  few  qualified  organic  resources  are  available  to  satisfy  the 
demand.  The  interviewees  believe  that  a  Government  center  could  retain  and  build 
technical  expertise  and  make  it  responsive  to  the  manager.  Appendix  B  further 
addresses  the  issue  of  training. 

Technical  Assessment  Services 

Technical  assessment  services  include  assessing  existing  AI  applications, 
assessing  AI  development  tools,  and  assessing  maturing  laboratory  AI  technologies. 
Interviewees  would  like  better  information  about  what  applications  other  DoD 
organizations  are  developing  to  determine  whether  a  particular  application  might 
have  merit  in  similar  functions  within  their  organizations.  Current  applications 
databases  contain  no  information  as  to  how  accurate,  complete,  well-documented, 
and  thoroughly  tested  a  system  is.  If  logistics  AI  systems  are  to  be  adapted  for  use 
throughout  DoD,  some  assessment  standards  or  confidence  rating  schema  must  be 
developed. 

Interviewees  also  suggested  confidence  rating  schemes  for  the  evaluation  of  AI 
development  tools.  Today,  new  tools  are  rapidly  entering  the  marketplace  and 
existing  tools  are  being  enhanced.  The  community  believes  that  a  central  g^oup 
tasked  with  assessing  new  tool  releases  and  notifying  the  users  of  any  substantial 
changes  in  functionality  and  performance  will  be  beneficial.  Currently,  this  tasking 
is  performed  redundantly  by  numerous  individuals  throughout  DoD. 

In  similar  fashion,  attempts  to  pull  maturing  AI  technologies  out  of  the 
laboratory  and  apply  them  to  logistics  activities  can  result  in  unnecessary 
duplication  of  effort  if  the  same  technology  evaluation  studies  are  being  conducted  or 
contracted  by  more  than  one  DoD  agency.  The  community  believes  that  the 
consolidation  of  these  activities  in  a  central  group  offers  a  streamlining  of  AI  activity 
and  will  permit  wider  and  more  thorough  investigations  of  emerging  AI  technologies 
than  might  otherwise  be  possible. 

Programming  Services 

The  fourth  most  requested  service  is  programming.  The  specific  progi'amming 
ser/ices  most  frequently  requested  were  applications  develop  nent  and  maintenance. 


test  and  evaluation,  configuration  management,  and  the  development  and  main¬ 
tenance  of  systems  documentation.  Individuals  frequently  expressed  the  idea  that 
some  personnel  at  their  locations  had  experience  with  specific  tools  and  access  to 
them.  However,  no  information  is  available  to  indicate  who  those  experienced 
personnel  are.  A  central  coordination  activity  with  information  on  what  tools  are 
available  within  the  organization  and  who  is  skilled  in  their  use  is  believed  to  be  of 
value  to  the  community. 

Communications  Services 

The  fifth  service  requested  was  communications.  Specific  functions  included  in 
this  area  are  electronic  bulletin  boards,  newsletters,  and  conferences.  The  consensus 
is  that  community  integration  will  serve  to  increase  the  awareness  of  existing 
capabilities  and,  hence,  reduce  the  duplication  of  effort. 

Interagency  communication  today  is  nindered  not  only  by  the  absence  of  means 
but  also  by  the  nonavailability  of  communications  standards.  For  example,  even  the 
language  used  to  describe  an  AI  system,  "a  prototype,”  is  vague.  A  RAND  document 
on  expert  systems  [25]  refers  to  four  different  prototypes  and  six  expert  system 
development  phases  (see  Figure  3-1).  The  AFLC  AI  PMO  regularly  communicates 
information  on  the  maturity  of  its  logistics  AI  application  projects  in  terms  of  four 
development  phases.  The  only  two  phases  in  which  common  terminology  is  used 
between  RAND  and  AFLC  are  the  terms  "concept  phase”  and  "development  phase.” 
However,  the  terms  as  used  by  AFLC  and  RAND  mean  completely  different  things 
depending  upon  whether  the  project  is  being  tracked  by  the  AFLC  AI  PMO  or  by  the 
RAND  methodology.  As  an  aside,  given  the  contracted  development  period  of  an  AI 
system  as  compared  to  the  development  timeframe  of  a  conventional  system,  four 
phases  of  development  are  recommended  as  adequate.  Communication  mechanisms 
and  terminology  should  be  determined  by  a  control  coordination  activity. 

Management  Support  Services 

Management  support  services  include  the  functions  of  performing  ROI 
analyses,  contracting  for  outside  services,  advocating  and  justifying  budgets  for 
project  seed  money  and  the  required  central  support  activities,  and  distributing  tools 
or  other  training  or  support  aids. 
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FIG.  3-1 .  RAND's  RECOMMENDED  EXPERT  SYSTEM  DEVELOPMENT  PROCESS 


Methodology  Services 

The  final  service  requested  by  the  logistics  community  during  the  interview 
process  was  in  the  area  of  methodologies.  Interviewees  requested  help  developing 
methodologies.  The  users  themselves  are  prepared  to  apply  the  methodologies. 
However,  they  believe  that  they  need  guidance  to  indicate  which  methodologies  can 
be  consistently  applied  in  the  areas  of  AI  systems  verification  and  validation, 
knowledge  engineering,  and  the  cost/benefit  analyses.  While  the  responses  in  this 
area  were  limited,  the  specific  methodologies  requested  are  not  well  understood 
within  the  AI  community.  The  DoD  Logistics  AI  Policy  Council  should  establish 
guidelines  in  these  areas  for  use  throughout  DoD. 

OTHER  GENERAL  REQUIREMENTS 

In  addition  to  identifying  specific  services  that  the  logistics  community  believes 
are  required  to  support  successful  AI  initiatives,  the  interview  process  also  uncovered 
general  opinions  about  the  form  and  structure  of  the  required  functions.  In  general, 
users  expressed  concern  that  the  functions  be  responsive,  on-site  or  reasonably 
accessible,  and  provide  for  limited  "no  cost  to  user”  services. 

Responsiveness 

Most  of  the  individuals  interviewed  stressed  the  importance  of  being  able  to 
"turn  on”  or  obtain  access  to  services  quickly.  For  information  services,  the  turn- 
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around  must  be  nearly  instantaneous.  For  longer  term  technical  support  or 
programming  services,  the  time  to  initiate  the  activities  must  be  no  more  than  a  few 
weeks. 

Presence 

A  concern  expressed  consistently  is  the  ability  to  deal  "personally”  with  a 
support  specialist  knowledgeable  in  both  logistics  and  AI.  Many  interviewees  believe 
that  some  form  of  on-site  presence  will  result  in  more  responsive  service.  Further, 
they  believe  it  is  important  that  their  time  not  be  wasted  having  to  explain  details  of 
the  logistics  process  to  the  individual  or  team  that  is  to  provide  the  support.  They 
believe  that  the  team  should  already  understand  the  important  functions,  leverage 
points,  and  interactions  of  the  logistics  process. 

"No  Cost  to  User"  Services 

Among  the  most  revealing  comments  are  the  desires  of  many  individuals  to 
obtain  limited  forms  of  support,  such  as  a  quick  answer  to  a  question  about  the 
advisability  of  using  a  particular  tool  for  a  particular  application  or  the  risks  of  one 
form  of  application  as  compared  to  another  in  a  different  functional  area  of  logistics. 
Many  individuals  who  wanted  to  begin  applying  the  technology  in  their  organization 
expressed  concern  that  funding  was  not  available  to  support  their  initiatives.  The 
OSD  Logistics  AI  Policy  Council  should  consider  supporting  a  limited  range  of 
services  to  be  provided  at  no  cost  to  the  user.  These  services  would  be  supported  by 
"core”  recurring  funding. 

RELATED  ISSUES 

Longevity 

The  DoD  logistics  community  requires  access  to  a  technically  and  functionally 
competent  and  experienced  staff.  The  OSD  support  function  should  provide  stability 
and  longevity. 

Flexibility 

While  stability  and  longevity  are  important,  the  OSD  logistics  AI  support 
function  must  be  responsive.  The  staff  should  be  free  to  expand  and  contract 
gracefully,  without  having  a  large  expensive  core  staff  to  support. 
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Independence 

After  speaking  with  numerous  individuals  from  different  organizations  across 
the  Military  Departments  and  Defense  agencies,  we  recommend  that  the  OSD 
Logistics  AI  support  function  remain  independent  of  any  particular  Military 
Department  or  agency.  That  independence  will  remove  any  potential  for  the  support 
activity  to  be  "limited”  by  association  with  a  particular  command  or  organization.  By 
promoting  independence,  OSD  can  ensure  all  DoD  organizations  feel  the  program  is 
supporting  their  requirements.  Further,  the  Military  Departments  and  Defense 
agencies  require  some  independence  to  execute  their  programs  within  the  guidelines 
and  policy  direction  outlined  by  the  Secretary  of  Defense. 

Government  Managed 

While  independence  is  important,  it  is  also  important  that  the  logistics  AI 
support  activity  be  directly  managed  by  a  government  body.  This  can  be  accom¬ 
plished  by  establishing  a  government  steering  group  to  continuously  evaluate  the 
OSD  production  and  logistics  AI  activities.  This  recommendation  is  directed  at 
eliminating  any  potential  for  "conflict  of  interest”  that  could  exist  if  the  DoD  logistics 
AI  support  activity  were  contractor  managed. 
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CHAPTER  4 


IMPLEMENTATION  OF  THE  DoD  ARTIFICIAL  INTELLIGENCE 

LOGISTICS  PROGRAM 


ORGANIZATIONAL  ISSUES 

One  of  the  key  lessons  learned  by  U.S.  industry  during  the  past  decade  in  its 
attempts  to  apply  AI  technologies  is  the  importance  of  establishing  activities 
dedicated  to  ensuring  the  organization  approaches  and  applies  the  technologies 
appropriately  to  realize  the  highest  leverage  to  the  mission.  Policy  guidance  and 
direction  must  be  established  and  should  be  provided  by  the  highest  levels  of 
management.  For  example,  both  DEC  and  IBM  have  top  executive  level  oversight 
committees.  In  addition,  the  services  discussed  in  Chapter  3  must  be  provided. 

Based  upon  a  careful  analysis  of  the  nature  and  requirements  of  the  AI 
technologies,  the  lessons  learned  by  industry  and  other  organizations,  and  the 
support  required  by  the  logistics  community  in  applying  AI  technologies  to  their 
functions,  we  recommend  the  organizational  structure  depicted  in  Figure  4-1  as  the 
best  structure  to  facilitate  the  rapid  implementation  of  AI  technologies  within  P&L- 
related  activities. 

DoD  Logistics  AI  Policy  Council 

The  Department  of  Defense  should  establish  a  Policy  Council  to  determine  the 
specific  direction,  priorities,  and  policies  of  its  logistics  AI  program.  The  DoD 
Logistics  AI  Policy  Council  should  be  chaired  by  the  ASD(P&L).  The  following 
individuals  should  be  designated  as  members  of  the  Council: 

•  Deputy  Assistant  Secretary  of  Defense  (Logistics)  [DASD  (L)] 

•  DASD  (Procurement) 

•  DASD  (Production  Support) 

•  DASD  (Installations) 

•  DASD  (Environment) 


FIG.  4-1 .  DoO  Al  LOGISTICS  PROGRAM  ORGANIZATIONAL  STRUCTURE 

•  DASD  (Systems) 

•  DASD  (Management  Systems) 

•  AI  Program  Manager,  Director  for  Logistics,  JCS 

•  AI  Program  Manager,  United  States  Air  Force 

•  AI  Program  Manager,  U nited  States  N avy 

•  AI  Program  Manager,  United  States  Army 

•  AI  Program  Manager,  United  States  Marine  Corps 

•  AI  Program  Manager,  Defense  Logistics  Agency. 

DoD  Logistics  AI  Program  Office 

In  addition,  the  ASD(P&L)  should  establish  a  DoD  Logistics  AI  Program  Office 
as  a  temporary  office  for  approximately  3  years.  The  DoD  Logistics  AI  Program 
Office  would  have  a  small  permanent  staff  of  six  that  would  be  supplemented  as 
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necessary  by  individuals  on  loan  from  the  Military  Departments  and  Defense 
agencies.  This  small  Program  Office  would  assist  the  Logistics  AI  Policy  Council  in 
establishing  and  implementing  policy  guidance  and  direction.  It  would  also  help 
establish  a  permanent  central  DoD  technical  support  staff  at  the  Logistics  AI 
Technology  Transition  Center  (TTC). 

Specifically,  the  DoD  Logistics  AI  Program  Office  would  be  tasked  to  perform 
those  management  functions  necessary  for  rapidly  transferring  AI  technologies  into 
logistics  operating  units  throughout  the  DoD  Components.  The  Program  Office  staff 
would  "differentiate”  and  push  the  application  of  AI  technologies  in  logistics  by 
pursuing  an  active  logistics  and  AI  marketing  effort.  That  effort  should  be  composed 
of  three  principal  activities;  publicity,  the  distribution  of  free  samples,  and 
presentations  to  carefully  selected  key  decision  makers  throughout  DoD.  The  staff 
should  widely  publicize  successful  logistics  AI  systems  and  should  emphasize  the 
real-world  benefits  that  the  systems  are  generating.  The  staff  should  also  publish 
articles  in  popular  logistics  journals,  including  the  Society  of  Logistics  Engineers 
Spectrum,  the  Military  Logistics  Forum,  etc.  The  Program  Office  should  distribute  to 
DoD  Components  free  samples  of  PC-based,  nontrivial  AI  systems  in  each  functional 
area  of  logistics  (maintenance,  inventory,  acquisition,  transportation,  warehousing, 
etc.).  Finally,  the  staff  should  brief  key  individuals  throughout  DoD  on  the  DoD 
Logistics  AI  Program.  The  staff  should  solicit  the  recommendations  of  those 
individuals  for  improving  the  OSD  initiatives.  The  Program  Office  should  establish 
the  goal  of  reaching  every  logistician  in  the  DoD  Components  and  making  them 
aware  of  the  potential  use  of  AI  technologies  in  logistics  before  the  office  is  officially 
disbanded. 

Logistics  AI  Technology  Transition  Center 

A  DoD  Logistics  AI  TTC  should  be  established  as  a  permanent  central  DoD 
activity.  It  should  be  tasked  to  provide  those  services  previously  identified  and 
discussed  in  Chapter  3  as  being  required  for  the  successful  implementation  of  an  AI 
program  (see  Table  3- 1 ). 

Analysis  of  the  user  requirements  data  and  interviews  with  those  commercial 
and  Federal  activities  that  have  successful  track  records  in  applying  AI  indicates  two 
overlapping  areas  of  requirements,  as  depicted  in  Table  4-1.  These  areas  are  the 
technical  disciplines  for  which  support  is  required,  e.g.,  expert  systems,  natural 
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TABLE  4-1 


TECHNOLOGY  TRANSITION  CENTER  SERVICES 


language,  speech,  etc.,  and  the  specific  types  of  services  that  are  required.  Table  4-1 
depicts  this  functional  framework  in  matrix  format.  This  matrix  should  be  used  to 
define  and  bound  the  TTC  concept. 

After  reviewing  the  feedback  from  the  logistics  AI  community,  we  examined 
alternative  means  of  providing  these  services.  One  means  is  the  creation  of  a  TTC. 
That  concept  has  a  precedent  in  the  form  of  the  more  than  20  chartered  DoD 
Information  Analysis  Centers  (lACs).  While  each  of  these  centers  is  dedicated  to  a 
particular  discipline  or  mission,  none  specifically  addresses  AI  or  AI  and  logistics. 

An  alternative  to  the  creation  of  a  TTC  is  the  establishment  of  a  new  LAC. 
However,  LACs  typically  collect,  review,  and  store  available  information  specific  to  a 
specialized  technical  discipline.  They  also  can  provide  a  network  of  technical  experts 
to  supplement  the  technical  information  with  consulting  in  specialized  areas  [26]. 

LAC  services  cover  the  information  services,  communications  services,  and  can 
cover  a  part  of  technical  support  services  (technical  consulting).  However,  LACs  do 
not  provide  the  other  services  and  functions  required  of  the  TTC  (see  Table  4-1). 
Hence,  we  decided  to  adapt  the  LAC  concept  and  recommend  a  TTC  specifically  suited 
to  the  transfer  of  AI  technologies  to  logistics. 

The  TTC  can  serve  as  a  central  DoD- wide  activity  capable  of  retaining  corporate 
memory  of  the  DoD  Logistics  AI  Program.  Further,  it  can  offer  a  set  of  services  at  no 
cost  to  the  user.  At  the  same  time,  the  TTC  can  operate  as  a  separate  cost  center 
capable  of  accepting  funding  for  extended  consulting,  evaluation,  or  programming 
services.  For  example,  a  criterion  of  20  hours  of  services  might  be  established.  Users 
would  only  be  charged  for  a  specific  project’s  services  that  exceed  20  hours  of  the  TTC 
staff  members’  time.  Thus,  the  TTC  could  provide  the  requested  "no  cost  to  the  user’’ 
services  and  at  the  same  time  maintain  basic  self-sufficiency.  A  small  recurring 
budget  would  be  required  to  support  the  ongoing  activities  of  the  TTC.  All  of  the 
corporate  central  AI  Application  Support  Centers  examined  in  Chapter  2  operate  as 
separate  cost  centers.  They  are  reimbursed  by  using  groups  throughout  the  company. 

Creation  of  a  DoD  Logistics  AI  TTC  does  not  preclude  either  the  decen¬ 
tralization  of  some  of  these  services  (e.g.,  programming)  to  Military  Department 
"mini-TTCs,"  nor  does  it  preclude  contracting  for  any  of  the  required  services. 
Successful  operation  of  the  TTC,  however,  does  provide  valuable  services  to  the  user 
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community  and  it  does  preclude  unnecessary  duplication  of  effort  and  the  waste  of 
resources. 

LOGISTICS  Al  POLICY  ISSUES  TO  BE  RESOLVED 

As  mentioned  throughout  this  report,  a  number  of  critical  policy  issues  must  be 
resolved  and  policy  guidance  provided  to  DoD  activities.  Table  4-2  summarizes  some 
of  those  issues.  The  Logistics  AI  Policy  Council  would  provide  a  forum  for  the 
discussion  and  resolution  of  these  difficult  issues.  Appendices  B,  C,  D,  and  E  provide 
specific  recommendations  and  frameworks  that  can  be  reviewed  and  used  to  resolve 
many  of  these  issues. 


TABLE  4-2 

AREAS  REQUIRING  POLICY  RESOLUTION 


Determination  of  specifications  and  standards 

•  Standard  tools/development  environment 

•  Terminology/database  fields  and  formats 

Acquisition  policy 

1 

•  Contracting  methodology 

•  In-house  development  versus  contract 

Commitment  of  resources 

Methodology  services 

•  Systems  verification  and  validation 

•  ROI/LCC  analyses 

•  Knowledge  engineering 

•  Technology  assessment 

•  Tool  assessment 

Systems  development  and  maintenance 

Application  qualification  guidelines 

Training  curricula 

•  In-house  training  versus  contract  training 

Note:  UCC  =  life-cycle  cost 
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ADP 

AES 

AFB 

AFLC 

AI 

AIAG 

AIASC 

AIBOD 

AIC 

AIMG 

AIPG 

AIPMO 

AIPO 

AISC 

AITC 

ASD 

ASM 

ATC 

BCS 

CALS 

CBR 

CDRL 

CEMS 


Automatic  Data  Processing 
Advanced  Engineering  Staff  (GM) 

Air  Force  Base 

Air  Force  Logistics  Command 
Artificial  Intelligence 

Artificial  Intelligence  Applications  Group  (DEC) 

Artificial  Intelligence  Applications  Support  Center  (AFLC) 
Artificial  Intelligence  Board  of  Directors  (DEC) 

Artificial  Intelligence  Center  (Lockheed) 

Artificial  Intelligence  Marketing  Group  (DEC) 

Artificial  Intelligence  Product  Group  (DEC) 

Artificial  Intelligence  Program  Management  Office  (AFLC) 
Artificial  Intelligence  Project  Office  (IBM) 

Artificial  Intelligence  Support  Center  (IBM) 

Artificial  Intelligence  Technologies  Center  (DEC) 

Assistant  Secretary  of  Defense 
Automated  Storage  Module 
Advanced  Technology  Center  (Boeing) 

Boeing  Computer  Services  (Boeing) 

Computer-Aided  Acquisition  Logistics  Support 
Chemical,  Biological,  Radiological 
Contract  Data  Requirements  List 
Comprehensive  Engine  Management  System 
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CEMSS 

CEO 


COLD 

CRLCMP 

CRMIC 

CSDG 

DEC 

DID 

DLA 

DMA 

DoD 

DSAC 

EDI 

EDS 

EIA 

ES 

FFP 

GE 

GFE 

GM 

GMAC 

GMR 

HOL 

LAC 

IBM 

ILS 

ILSP 


=  Current  Engineering  and  Manufacturing  Services  Staff  (GM) 
=  Chief  Executive  Officer 

=  Conference  of  the  Logistics  Directors 

=  Computer  Resources  Life  Cycle  Management  Plan 
=  Corporate  Robotics  and  Machine  Intelligence  Council  (GM) 

=  Configuration  Systems  Development  Group  (DEC) 

=  Digital  Equipment  Corporation 

=  Data  Item  Descriptions 

=  Defense  Logistics  Agency 

=  Dues  Management  Advisor 

=  Department  of  Defense 

=  Defense  Logistics  Agency  Systems  Automation  Center 
=  Electronic  Data  Interchange 

=  E  lectronic  Data  Systems  (GM) 

=  Electronic  Industries  Association 

=  Expert  System 

=  Firm  Fixed  Price 

=  General  Electric 

=  Government  F  urnished  Equipment 

=  General  Motors  Corporation 

=  General  Motors  Acceptance  Corporation  (GM) 

=  General  Motors  Research  Laboratories  (GM) 

=  Higher  Order  Language 

=  Information  Analysis  Center 

=  International  Business  Machines  Corporation 
=  Integrated  Logistics  Support 

=  Integrated  Logistics  Support  Plan 


Gloss.  2 


ISTG 


JCS 

KBS 

KEE 

LAICC 

LCC 

LOGPARS 

LSAR 

NDI 

OSD 

P&L 

PC 

PECI 

PIF 

PMO 

RAMP 

ROI 

SOW 

STL 

TACOM 

TSG 

TTC 

TQM 

WIP 


Intelligent  Systems  Technology  Group 

Joint  Chiefs  of  Staff 

Knowledge-Based  System 

Knowledge  Engineering  Environment 

Logistics  Artificial  Intelligence  Coordinating  Cell 

Life-Cycle  Cost 

Logistics  Planning  and  Requirements  Simplification 

Logistics  Support  Analysis  Record 

Nondestructive  Inspection 

Office  of  the  Secretary  of  Le^ense 

Production  &  Logistics 

Personal  Computers 

Productivity  Enhancing  Capital  Investment 
Productivity  Investment  Funding 
Program  Management  Office 
Rapid  Acquisition  of  Manufactured  Parts 
Return  On  Investment 
Statement  of  Work 

Santa  Teresa  Program  Laboratories  (IBM) 

Tank  and  Automotive  Command  (Army) 

Technical  Staffs  Group  (GM) 

Technology  Transition  Center 
Total  Quality  Management 
Work  In  Process 
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APPENDIX  A 


CORPORATE  ORGANIZATIONS  APPLYING 
ARTIFICIAL  INTELLIGENCE 

DIGITAL  EQUIPMENT  CORPORATION 

Digital  Equipment  Corporation  (DEC)  is  a  computer  manufacturing  company 
whose  product  is  a  complex  mix  of  hardware  and  software  configured  to  meet  an 
individual  customer’s  needs.  DEC’S  marketing  distinctiveness  comes  from  the 
flexibility  it  allows  its  customers  in  component  selection.  Rather  than  marketing 
standard  systems  with  a  limited  number  of  options,  DEC  markets  customer-tailored 
systems.  It  believes  that  this  a  la  carte,  build-to-order  marketing  strategy  for  big 
computer  systems  is  a  competitive  advantage.  Hardware  and  software  configuration 
is  at  the  core  of  this  strategy.  The  product  line  consists  of  42  different  families  of 
central  processor  types  and  their  peripherals  and  software. 

DEC  got  its  start  in  artificial  intelligence  (AI)  because  of  a  very  important 
business  problem  that  had  senior  management  recognition  and  resolution  support. 
Configuring  a  computer  system  tailored  to  a  customer’s  need  is  a  time-consuming  and 
error-prone  activity  that  is  dependent  on  the  talent  and  skill  of  an  experienced 
configurator.  Since  such  talent  is  rare  and  expensive,  DEC  decided  to  develop  an 
expert  system  configurator  in  order  to  maintain  its  build-to-order  marketing  strategy 
[A-1  and  A-2]. 

The  corporate  culture  in  DEC’s  manufacturing  operations  is  open  to  change  and 
new  ideas  and  technologies  that  will  help  it  in  its  corporate  mission.  Carrying  ideas 
across  functional  boundaries  is  an  acceptable  and  encouraged  practice.  Conse¬ 
quently,  it  obtained  its  initial  AI  expertise  through  university  channels  with 
discretionary  funds,  under  the  sponsorship  of  a  group  engineering  manager  and  the 
Vice  President  of  Research  [A-3].  DEC  hired  John  McDermott,  a  professor  at 
Carnegie-Mellon  University,  to  develop  a  configuration  expert  system  prototype, 
called  Rl,  for  the  VAX  11/780  computer  [A-3  and  A-4].  After  this  successful  proof-of- 
concept,  the  management  issues  became  how  to  integrate  Rl  with  as  little  disruption 
as  possible,  and  how  to  establish  a  group  able  to  continue  developing  and  extending 


Rl  [A-4].  With  the  sponsorship  of  the  Vice  President  of  Manufacturing  and  the  Vice 
President  of  U.S.  Manufacturing,  the  Configuration  Systems  Development  Group 
(CSDG)  was  established  in  1980  [A-2]. 

The  first  DEC  corporate-sponsored  AI  training,  support,  and  apprenticeship 
programs  were  taught  by  the  CSDG  and  paid  for  by  Manufacturing.  In  1984,  the 
Artificial  Intelligence  Technologies  Center  (AITC)  was  formed.  Initially,  it  reported 
to  Manufacturing.  Early  coordination  and  planning  across  functional  groups  was 
informal,  a  product  of  DEC’s  corporate  culture.  Recently,  coordination  needed  to 
tackle  the  configuration  issues  has  been  formalized  in  a  Configuration  Systems 
Steering  Committee  of  managers  of  strategically  selected  business  constituencies 
[A-2]. 

The  AITC  is  no  longer  in  the  Manufacturing  function  but  instead  reports  to  the 
Artificial  Intelligence  Board  of  Directors  (AIBOD),  chaired  by  the  DEC  Vice 
President  of  Networks  and  Distributed  Systems  Group  and  composed  of  Vice 
Presidents  from  Engineering,  Marketing,  Manufacturing,  Software  Services,  and 
Field  Services.  The  AIBOD  reports  to  the  Senior  Vice  President  in  charge  of  all 
product-related  operations,  who  in  turn  reports  directly  to  the  Chief  Executive 
Officer  (CEO).  The  mandate  for  the  AITC  is  permanent  [A-2]. 

The  AITC  has  four  groups;  The  Intelligent  Systems  Technology  Group  (ISTG) 
develops  and  maintains  DEC’s  internal  apnlications;  the  Artificial  Intelligence 
Applications  Group  (AIAG)  consists  of  developers  of  internal  diagnostic  applications 
for  DEC  field  service  organizations;  the  Artificial  Intelligence  Product  Group  (AIPG) 
includes  the  people  responsible  for  selling  DEC’s  AI  products  (VAX  Lisp,  VAX  OPS5, 
and  the  AI  VAXStation);  and  the  Artificial  Intelligence  Marketing  Group  (AIMG) 
markets  DEC’s  AI  hardware,  AI  software,  and  AI  consulting  and  training  services 
[A-.5]. 

The  AITC  is  further  integrated  with  the  rest  of  DEC’s  organization  via  a  matrix 
of  reporting  responsibilities.  Selected  individuals  in  various  business  constituencies 
(Engineering,  Manufacturing,  Field  Service,  Software  Services,  Personnel, 
Marketing,  and  Finance)  also  report  directly  to  the  Manager  of  AITC.  For  example, 
Ms.  Virginia  Barker  is  Manager  of  CSDG  in  Manufacturing  and  as  such  reports  to 
the  Vice  President  of  Manufacturing.  She  is  also  in  the  ISTG  part  of  the  AITC,  and 


reports  to  the  Manager  of  the  AITC.  (Recall  that  the  Vice  Presidents  of  many  of  these 
business  constituencies  also  sit  on  the  AJBOD.)  [A-2] 

The  distribution  of  AI  tasks  at  DEC  is  both  centralized  and  distributed.  The 
Intelligent  Systems  Technology  Group,  the  AI  Applications  Group,  the  AI  Product 
Group,  and  the  AI  Marketing  Group  have  all  been  collected  under  the  AITC  [A-5]. 
Thus,  AI  engineering  development,  validation,  and  disbursement;  hardware  and 
software  selection;  and  external  marketing  functions  are  centralized.  Knowledge 
engineering  activities,  maintenance  of  production  systems,  and  some  forms  of 
applications  support  are  line-function  responsibilities  [A-2]. 

Training  is  conducted  at  the  AITC  in  Marlboro,  MA.  DEC  business  units  pay 
for  personnel  and  materials  and  supply  the  domain  and  integration  expertise  —  the 
AITC  with  more  than  200  people  is  a  cost  center  [A-5].  Application  selection 
expertise  is  supplied  by  the  AITC.  Applications  support  is  organic,  coming  from 
either  line-function  personnel  or  the  AITC  [A-2].  Worldwide,  DEC  has  approx¬ 
imately  500  people  working  in  AI,  half  of  them  in  line-functions  on  internal  projects 
[A-2]. 

Program  payback  has  not  been  rigorously  measured.  A  general  estimate  is 
$100  million  per  year  [A-2].  DEC’S  AI  efforts  in  configuration  have  allowed  the 
corporation  to  maintain  a  marketing  strategy  that  handles  increasing  product 
complexity  and  preserves  customer  choice. 

The  experience  has  had  an  unexpected  payback  in  its  effect  on  DEC’s  strategic 
view  of  its  business.  The  process  has  given  rise  to  the  knowledge  network  vision 
based  on  the  idea  of  the  "knowledge  network.”  The  knowledge  network  is  "the 
everyday  problem-solving  activity  within  the  organization  (that)  can  be  thought  of  as 
conducted  by  a  network  of  experts  knowledgeable  about  the  products  and  the  physical 
and  paperwork  processes  that  constitute  the  business”  [A-6].  Within  the  knowledge 
network  are  "pockets  of  expertise”  with  decision-making  know-how  and  undoc¬ 
umented  expertise.  These  "pockets  of  expertise”  are  order  administration, 
engineering,  planning,  manufacturing,  production,  distribution,  marketing  and 
sales,  customer-field  support,  and  the  customer.  Point  solutions  fail  to  make  this 
connection  and  miss  this  higher  level  of  integration. 

Two  major  business  cycles  appear  within  DEC’s  knowledge  network  —  the 
order-process  cycle  and  the  product  life  cycle.  The  order-process  cycle  involves 


product  requirements,  specifications  and  design,  intelligent  manufacturing,  and 
order  logistics  such  as  scheduling  and  sourcing.  The  product  life  cycle  involves 
developing  product  requirements,  transitioning  into  manufacturing,  planning  the 
manufacturing  process,  and  managing  projects. 

The  knowledge  network  vision  has  led  to  the  development  of  these  order-process 
cycle  expert  systems:  XCON  is  used  to  validate  the  configurability  of  customer 
orders  and  to  guide  actua  assembly  of  these  orders.  XSEL  is  used  interactively  to 
assist  in  the  selection  of  the  saleable  parts  that  make  up  a  customer’s  order.  XFL  is 
used  to  diagram  a  computer  room  floor  layout  for  the  configuration  under 
consideration.  XCLUSTER  is  used  to  assist  in  configuring  clusters  (DEC  sells  its 
VAX  computer  line  in  resource-sharing  clusters  of  central  processing  units).  XNET 
is  an  expert  system  that  will  design  and  configure  local  area  networks.  The 
knowledge  network  vision  has  led  to  the  development  of  these  product  life-cycle 
expert  systems;  SIZER  is  a  research  effort  in  customer  computer  resource  planning. 
RIME  is  a  software  engineering  methodology  to  enable  the  more  effective  building  of 
new  configuration  expert  systems  [A-6].  The  knowledge  network  vision  fosters  the 
mechanisms  for  cross-functional  communication  and  feedback.  The  payback  to  DEC 
includes  manufacturing  process  efficiencies,  specification  change  enablement,  evolu¬ 
tion  to  future  processes,  and  a  testbed  for  software  product  technologies. 

The  first  big  lesson  for  DEC  is  that  it  needs  a  critical  mass  of  expertise.  As 
pointed  out,  the  AITC  has  more  than  200  members  at  its  Marlboro  location.  Its 
activities  center  around  DEC’s  needs  with  AI:  manufacturing  and  engineering 
applications,  AI  product  development  (VAXLisp,  VAX  OFS5,  AI  VAXStation),  and 
marketing  of  DEC  AI  products  and  services.  The  second  lesson  for  DEC  is  that  it 
needs  a  mechanism  to  transfer  the  technologies  into  the  business  units  [A-2]. 

INTERNATIONAL  BUSINESS  MACHINES  CORPORATION 

International  Business  Machines  (IBM)  is  one  of  the  world’s  largest 
corporations.  Its  business  involves  every  area  of  advanced  technology  in  information 
processing,  from  semiconductors  to  superconductors.  Research  and  development  and 
product-related  engineering  efforts  cost  more  than  $5  billion  a  year  (IBM  Innovation. 
July  1987). 

In  1985,  a  report  to  the  Corporate  Management  Committee  declared  that  AI 
was  mature  enough  for  IBM,  and  recommended  that  the  company  establish  a  small. 


high-level  group  with  access  to  top  management  to  coordinate  IBM’s  application  of 
the  technologies.  Following  that  recommendation,  IBM  made  a  top-level  manage¬ 
ment  decision  to  use  AI  in  1985.  Its  Corporate  Management  Committee,  comprised  of 
the  CEO,  Chairman  of  the  Board,  and  several  Vice  Presidents,  tasked  the  Vice 
President  of  the  Systems  Research  Division  to  head  the  effort  [A-3].  The  Vice 
President  of  Systems  established  the  Artificial  Intelligence  Steering  Committee  as 
part  of  the  Corporate  Management  Board.  The  Corporate  Management  Board  is 
made  up  of  approximately  25  senior  level  corporate  officers  and  reports  to  the 
Corporate  Management  Committee.  At  the  recommendation  of  the  Vice  President  of 
Systems  Research  Division,  the  Management  Committee  created  the  Artificial 
Intelligence  Project  Office  (ATPO)  [A-7]. 

Before  1985,  IBM  made  some  effort  to  apply  AI  to  existing  problems.  One  of  the 
first  tasks  of  the  IBM  AIPO  was  to  listen  to  what  the  line-functions  were  doing  in  AI. 
The  Project  Office’s  objectives  were  to  solidify  IBM’s  project  plan  for  expert  systems, 
to  make  IBM’s  marketing  of  expert  systems  a  relative  success  in  the  United  States 
and  worldwide,  and  to  develop  an  internal  AI  use  program. 

The  Project  Office  was  established  as  a  temporary  organization  and  officially 
disbanded  at  the  end  of  June  1989.  The  office  was  intentionally  small.  Only 
14  persons  provided  top-level  strategic  coordination.  The  Project  Office  was  part  of 
Enterprise  Systems  reporting  directly  to  the  Corporate  Management  Board. 

The  AIPO  successfully  imbued  IBM’s  infrastructure  with  the  ability  to 
distribute  skills  to  support  expert  systems  development  and  fielding.  The  two  major 
thrusts  were  technology  transfer  and  distributed  development  [A-7].  The  office 
succeeded  in  part  by  providing  seed  money  for  strategic  projects.  Ongoing 
responsibilities  for  technology  and  AI  technology  transfer  have  now  been  transferred 
to  Programming  Systems.  Ongoing  responsibilities  for  AI  policy  have  been 
transferred  to  Corporate  Information  Systems. 

The  AISC  actually  predates  the  AIPO  and  is  part  of  the  Programming  Systems 
line-of-business.  As  part  of  the  Santa  Teresa  Program  Laboratories  (STL),  the  AISC 
was  physically  moved  in  1986  to  STL’s  Palo  Alto  Research  Laboratories  to  be  close  to 
the  Menlo  Park  Development  Laboratories  where  knowledge-based  systems  (KBS) 
development  under  STL  is  located.  AISC’s  mission  is  to  provide  KBS  application 
support  for  all  IBM  operating  units  [A-8].  AISC  programs  include  application 
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support  of  grass  roots  projects  in  the  form  of  consultation  and  collaborative  projects; 
KBS  education,  from  fundamentals  to  advanced  functions,  and  executive  and 
technical  trainings;  information  exchange  and  lessons  learned  through  an  internal 
use  database,  site  support  groups,  and  site  user  groups;  technology  support  in  tools 
and  languages;  and  advanced  technical  enhancements  (image,  speech,  etc.). 

AISC  uses  the  AI  Internal  Use  Council  to  internally  market  AI  technologies  to 
IBM’s  lines-of-business  and  communicate  success  stories.  This  marketing  effort  is 
part  of  the  attempt  to  distribute  the  technology.  AISC’s  objective  is  to  establish 
centers  of  competency  within  IBM’s  lines-of-business. 

Between  an  estimated  600  and  800  internal  customers  are  actively  developing 
projects.  Excluding  marketing  and  research,  13  locations  and  75  to  100  individuals 
provide  some  kind  of  support  for  the  internal  use  of  AI  within  IBM  worldwide. 
Corporate  Information  Services  takes  the  policy-making  position  vacated  by  the 
termination  of  the  AIPO  and  will  determine  future  corporate  policy  for  IBM  internal 
use. 

The  estimated  return  for  commercial  operations  is  greater  than  $50  million  per 
year.  From  the  strategic  view,  the  AIPO’s  objectives  are  a  consistent  and  worldwide 
marketing  of  AI  technologies  and  a  coherent  product  (expert  systems  tools)  line. 
Also,  from  a  market  point  of  view,  positioning  the  company  to  enable  product 
evolution  from  high-volume  data  processing  into  intelligent  data  processing  products 
(from  automation  of  back  office  functions  to  automation  of  front  office  functions)  is  a 
principal  benefit  [A-7]. 

BOEING  CORPORATION 

Boeing’s  business  encompasses  the  engineering,  design  and  manufacturing  of 
airplanes,  helicopters,  aerospace  technologies,  and  commercial  and  military  avionics. 
In  1988,  Boeing  Commercial  Aircraft  delivered  290  jet  transports  and  337  other 
commuter  aircraft.  Boeing  Computer  Services  (BCS)  is  an  integrator  of  large-scale, 
complex  information  and  telecommunications  systems,  with  broad  activities  in 
information  and  telecommunications  technologies  [A-9]. 

Boeing  entered  into  AI  in  response  to  external  research  and  development 
requests  for  proposals  from  the  Government.  Projects  within  Boeing  are  related  to 
each  division’s  business  needs.  The  Center  for  AI  is  in  the  Advanced  Technology 


Center  (ATC)  for  Computer  Sciences  (formerly  the  Advanced  Technology 
Applications  Division)  of  BCS  [A-10]. 


During  the  startup  phase,  the  ATC  conducted  all  AI  training,  including 
everything  from  beginning  courses  in  AI  to  graduate-level  course  work.  The  ATC 
would  take  a  problem  domain  and  line-function  personnel  and  work  in  a  consulting 
relationship  until  the  line-function  personnel  could  take  a  solution  back  to  the  home 
division.  The  ATC  handled  both  training  and  technology  transfer  into  the  divisions. 

Subsequently,  training  has  been  transferred  into  the  Education  and  Training 
Divisions  and  visits  by  ATC  associates  are  sponsored  by  the  home  divisions.  BCS 
Division,  Boeing  Aerospace  and  Electronics  Division,  Boeing  Military  Airplane, 
Boeing  Advanced  Systems,  and  Boeing  Helicopter  all  have  support  divisions  or 
groups  with  AI  capabilities. 

Some  AI  te  hnologies  have  been  transferred  to  divisions  that  can  best  use  and 
fund  them;  for  example,  machine  vision  and  robotics  were  transferred  to  Boeing 
Aerospace.  Internal  research  and  development,  consulting,  and  technical  assistance 
support  are  functions  of  the  Center  for  AI. 

A  Communications  Manager  in  ATC  is  responsible  for  colloquia  with 
universities,  guest  speaker  visits,  AI  forums  open  to  all  employees,  an  annual 
symposium  on  AI,  and  monthly  ATC-wide  meetings.  Boeing’s  infrastructure 
incl'.ides  electronic  mail  and  bulletin  boards. 

The  Center  for  AI  contributes  to  the  corporate  vision  for  the  1990s.  Team 
members  from  the  center  serve  on  BCS’s  future  standards  committees.  For  example, 
BCS  sponsors  an  Architecture  Committee,  with  subcommittees  chartered  to  develop 
standards.  Team  members  serve  on  the  Software  Engineering,  Hardware,  and  KBS 
Engineering  subcormnittees. 

The  Center  for  AI  has  more  than  100  staff  members  and  is  organized  into  teams. 
A  management  unit  comprised  of  the  General  Manager,  Chief  Scientist,  and 
laboratory  staff  sets  priorities  for  external  requests,  internal  requests,  and  internal 
research  and  development.  The  AI  skill  base  is  also  distributed  across  Boeing’s 
operating  companies. 
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FMC  CORPORATION 


FMC  is  an  international  conglomerate  with  manufacturing  and  mining 
facilities  worldwide,  agricultural  equipment  and  chemical  concerns,  and  defense 
equipment  and  research  and  development  activities. 

In  late  1982,  the  Defense  Systems  Group  was  working  on  autonomous  vehicle 
technologies  as  part  of  its  land  combat  vehicle  business.  Because  of  lessons  learned 
from  the  work  in  vision  and  autonomous  control,  the  broad  applicability  of  AI  to 
FMC’s  other  businesses  —  agriculture,  industrial  chemicals,  and  commercial 
equipment  —  received  recognition  and  support  from  top  management.  In  1984,  the 
CEO  and  the  President  and  Chief  Operations  Director  of  FMC  decided  to  make  a 
major  investment  in  AI  as  strategic  technologies  [A-11]. 

FMC  decided  to  build  a  central  group  for  applied  AI  research  and  development. 
The  Artificial  Intelligence  Center  (AIC)  was  funded  as  part  of  the  Electronics, 
Engineering  and  Computer  Sciences  Laboratories  in  FMC’s  Central  Engineering 
Laboratories  in  Santa  Clara,  California. 

The  AIC’s  objective  was  to  be  a  leader  in  the  application  of  AI  in  industry  and 
defense.  The  emphasis  was  on  immediate  and  repeating  application  needs.  Further, 
FMC  selected  a  subset  of  AI  technologies  as  applicable  to  computing  between  then 
(1984)  and  2000.  The  selected  technologies  were  vision,  knowledge-based  planning 
and  configurations  systems,  structured  selection  and  data  interpretation,  intelligent 
human/machine  interface,  and  computer-based  training.  Other  objectives  were  to 
develop  the  technical  "integration”  skills  —  both  cultural  and  electronic  —  to 
successfully  transfer  the  technology  to  the  divisions,  and  to  leverage  internal 
capabilities  through  affiliations  with  local  universities  (e.g.,  Stanford  University) 
and  companies  (FMC  purchased  10  percent  of  Teknowledge,  an  expert  systems  tool 
vendor). 

The  AIC  was  funded  through  division-  and  corporate-sponsored  work.  Research 
and  development  and  capability  development  were  budgeted;  operating  divisions  and 
Government  contracts  paid  for  applications  development. 

Personnel  were  trained  for  internal  development  of  the  AIC  and  not  for 
technology  transfer  to  operating  divisions  and  dissemination  of  the  technologies.  The 


plan  was  to  recruit  talent  for  the  center  by  offering  an  exceptionally  attractive 
environment. 


Divisions  are  responsible  for  life-cycle  support  of  applications.  The  AIC 
transferred  development  technologies  and  integration  skills  to  the  divisions  in  the 
course  of  applications  development. 

By  1987,  the  AIC  had  about  40  professionals,  80  percent  of  whom  held  advanced 
degrees,  half  of  which  were  Ph.D.s.  In  February  1989,  the  AIC  was  broken  into 
subgroups:  an  AI  Department  tasked  with  applied  research  and  applications 
development;  a  Software  Engineering  Group  combining  knowledge  engineers  and 
conventional  software  programmers;  and  a  Signal  Processing,  Vision  and  Electronics 
Group,  which  handles  vision  technology  needs.  All  were  located  in  the  Electronics, 
Engineering  and  Computer  Sciences  Laboratories.  The  AIC  no  longer  trains  new 
personnel;  it  now  considers  itself  fully  staffed. 

An  additional  model  for  technology  dissemination  among  the  divisions  is  being 
implemented.  FMC  is  distributing  low-end  expert  systems  tools  throughout  the 
divisions  in  an  attempt  to  disperse  the  expertise  broadly.  The  Central  Engineering 
Laboratories  conduct  regular  seminars  for  division  personnel;  AI  topics  are  included. 
It  also  conducts  a  monthly  5-day  KBS  course  in  conjunction  with  the  distribution  of 
low-end  tools. 

Of  70  to  80  total  applications  to  date,  an  estimated  25  have  been  fielded 
(including  defense  demonstration  application  prototypes  built  under  contract). 

Evaluation  of  payback  is  informal.  Historically,  no  postproject  audits  were 
performed;  hence,  no  corporate- wide  figures  are  available.  Projects  have  succeeded  in 
capturing  vanishing  expertise  from  retiring  personnel,  and  anecdotal  stories  tell  of 
increased  performance  and  financial  returns. 

E.  I.  DU  PONT  DE  NEMOURS  AND  COMPANY 

Du  Pont,  with  annual  revenues  of  more  than  $29  billion,  is  a  major  American 
company  in  both  the  chemical  and  oil  industries.  As  a  modern  conglomerate,  it  has 
grown  from  its  1802  origin  to  produce  agricultural  chemicals,  consumer  products, 
biomedicine,  transportation  products,  and  plastics.  The  consumer  experiences 
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Du  Font’s  products  daily  —  Teflon,  nylon,  freon  gas,  T5rvek  envelopes,  and  Philips 
optical  disks  to  name  a  few. 

Du  Font’s  entry  into  AJ  technologies  was  an  outcome  of  its  corporate  planning 
office’s  function  of  investigating  new  technologies.  In  1985,  Dr.  Edward  Mahler  was 
a  liaison  to  emerging  technologies  in  the  Planning  Department.  After  evaluating  the 
technology  and  available  commercial  expert  systems  shells,  he  concluded  that  the 
ante  was  too  high  —  the  cost  of  acquiring  custom  expert  systems  tools  for 
implementing  custom  solutions  would  be  too  great.  To  finish  his  evaluation. 
Dr.  Mahler  selected  a  few  problems  and  built  a  few  simple  prototype  systems  with 
simple  tools.  That’s  when  he  struck  upon  a  simpler,  less-expensive  approach. 

Dr.  Mahler  reasoned  that  Du  Pont  had  thousands  of  personal  computer 
(PC)  -literate  users  and  nearly  as  many  PCs.  Even  though  the  company  had  several 
thousand  programmers,  probably  only  a  handful  could  program  in  any  of  the  expert 
systems  languages  —  LISP,  PROLOG,  OPS,  and  the  like.  Furthermore,  Du  Font’s 
distribution  over  more  than  120  sites  worldwide  did  not  lend  itself  to  cultivating  a 
whole  new  way  of  programming.  Therefore,  the  solution  had  to  be  simple  enough  to 
be  implemented  by  the  using  expert,  had  to  run  on  existing  hardware,  and  had  to 
address  a  user’s  real  need.  The  plan  became  to  provide  a  small  (able  to  run  on 
existing  PCs),  simple  (usable  by  nonprogrammers),  standard  tool  with  much  of  the 
appropriate  features  built-in,  and  help  the  user  develop  and  maintain  his  or  her  own 
application. 

Du  Font’s  executive  committee  agreed  to  fund  an  AI  Office  to  catalyze  the 
implementation  of  AI,  particularly  KBS  broadly  and  efTectively  "throughout  Du 
Pont’’  [A-12].  Du  Font’s  corporate  culture  recognizes  the  fact  that  the  company  has 
operations  at  more  than  a  hundred  sites  around  the  world.  Management  tries  to 
foster  informal  and  formal  networks  as  a  means  for  communicating  ideas  and 
solutions.  Because  operations  are  spread  throughout  the  world,  the  best  way  a 
corporate  group  can  help  is  to  support  the  operating  groups  by  serving  as  enablers. 
Rather  than  "pushing"  the  technology,  the  AI  Office  tries  to  create  awareness, 
motivation,  and  functional  capability  in  expert  systems  [A- 13]. 

The  Du  Pont  AI  Office  has  12  staff  members.  Its  training  tasks  are  to  provide 
awareness  courses  for  management,  skill-building  courses  for  potential  developers 
(the  technical  domain  experts),  and  instruction  on  the  use  of  simple  PC-based  shells. 
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The  AI  Office  staff  operates  a  telephone  hot-line  and  also  participates  in  "jump 
starts"  —  joint  rapid  prototyping  sessions  of  short  duration.  It  sponsored  the  creation 
of  an  in-house  expert  systems  shell  —  Tool  Kit  —  because  existing  commercial  shells 
were  deemed  too  difficult  for  nonprogrammers  to  use.  Tool  Kit  requires  no 
programming  knowledge,  has  built-in  graphics  and  statistics,  and  is  easy  enough  to 
use  to  be  taught  in  a  2-day  course.  The  AI  Office  staff  also  teaches  PC  MS-DOS  tools 
obtained  under  site  license  —  Insight  2-1-  and  Ist-CLASS  [A-3].  Thus,  the  staff 
provides  standard  tools  and  operating  units  use  them  to  implement  custom  solutions. 
The  site  license  lowers  the  operating  unit'c  tool  cost. 

Du  Font’s  AI  Office  also  has  a  technologies  assessment  function.  The  group 
recognizes  that  different  expert  systems  tools  are  needed  for  solving  different  classes 
of  problems,  and  potentially  big  problems  might  require  complex  solutions.  To  this 
end,  the  AI  Office  supports  a  few  higher  level  tools  and  languages. 

Du  Pont  has  more  than  200  systems  in  routine  use  and  600  under  development 
or  in  field  test  F A-3]. 

The  average  payback  is  estimated  at  between  7:1  [A-3]  and  15:1  [A-13].  The 
investments  are  small  and  the  returns  on  investment  (ROD  are  comparable  to  those 
experienced  by  other  companies  applying  the  technologies.  Du  Pont  also  counts  other 
forms  of  payback.  Du  Pont  refers  to  one  type  of  application  as  "principal  consultant 
apprentice"  expert  systems.  The  principal  consultant  apprentice  expert  system 
handles  80  percent  of  the  end-user  problems,  freeing  the  expert  to  work  on  the 
remaining  20  percent.  Also,  through  replication  of  expertise,  Du  Pont  has  measured 
improved  quality  from  nonexperts.  Another  benefit  is  consistency  of  decision 
making.  Finally,  Du  Pont  recognizes  expert  systems  as  a  means  for  documenting 
technology  and  preserving  corporate  expertise  [A-3]. 

LOCKHEED  CORPORATION 

Lockheed  Corporation  has  five  major  groups:  Missiles  and  Space  Systems 
Group  (including  Lockheed  Missiles  and  Space  Company,  Inc.;  Lockheed  Engineering 
and  Management  Services  Company,  Inc.;  and  Lockheed  Space  Operation  Company); 
Aeronautical  Systems  Group  (including  Lockheed  California  Company,  Lockheed 
Georgia  Company,  and  Lockheed  Aircraft  Service  Company);  Marine  Systems  Group 
including  Lockheed  Shipbuilding);  Electronics  Group  (including  Sanders  Associates, 
Inc.;  and  Lockheed  Electronics  Company,  Inc.);  and  Information  Systems  Group 
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(including  Cadiun,  Inc.;  Dialog  Information  Services,  Inc.;  Datacom  Systems 
Corporation;  Lockheed  Air  Terminal,  Inc.;  Metier  Management  Systems;  and  Cal 
Comp  Products).  Lockheed’s  Artificial  Intelligence  Center  (AIC)  is  in  the  Research 
and  Development  Division  of  Lockheed  Missiles  and  Space  Company,  Inc. 

In  late  1985,  a  Lockheed  corporate  AI  task  force  determined  a  need  and  made 
the  recommendation  for  an  AIC.  Within  6  months,  a  manager  had  been  assigned  in 
the  Research  and  Development  Division.  The  first  task  of  the  AIC  was  to  survey 
Lockheed’s  organizations  and  make  a  situational  assessment.  From  this  assessment 
came  the  AIC’s  charter. 

The  AIC’s  charter  has  three  major  areas:  research  and  technology  development, 
technology  transfer,  and  education  and  training.  The  AIC  is  organized  into  nine 
sections.  The  seven  sections  working  on  research  and  technology  development  are: 
the  Validation  of  Knowledge  Bases  Section,  the  Human  Machine  Interface  Section, 
the  Real  Time  Symbolic  Systems  Section,  the  Advanced  Reasoning  Systems  Section, 
the  Generic  Expert  Systems  Section,  the  Image  Understanding  Section,  and  the 
Autonomous  Planning  Section.  The  two  remaining  sections  are  the  Technology 
Transfer  Section  and  the  Training  and  Education  Section. 

In  addition  to  research  in  the  particular  areas,  the  sections  develop,  test,  verify, 
and  support  AI  tools,  some  in-house  and  some  commercial.  The  AIC  supports 
university  research  projects  and  has  connections  with  Stanford  University.  The  AIC 
also  has  affiliations  with  commercial  organizations  (e.g.,  Inference  Corp.  and  SRI 
International)  to  help  leverage  AI  technologies.  In  addition,  the  sections  conduct 
externally  funded  research. 

The  Technology  Transfer  Section  is  tasked  with  establishing  technology 
transfer  interfaces  with  Lockheed’s  product  divisions.  It  conducts  the  AIC’s  technical 
(consulting  with  product  divisions,  acts  as  the  conduit  to  integrate  the  AIC’s  AI 
research  into  the  product  divisions,  and  maintains  the  corporate  AI  applications 
database. 

The  Training  and  Education  Section  conducts  short  courses  of  1  to  2  weeks  in 
implementation  skills  and  holds  l-day  to  1-week  courses  for  executives;  those  courses 
include  an  overview  of  AI  technologies  and  AI  project  management  techniques.  The 
Training  and  Education  Section  also  has  a  6-month  residency  program.  In  that 
program,  a  product  division  nominates  a  candidate,  approves  a  project,  and  submits  a 
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proposal  to  the  AIC.  The  resident  attends  12  weeks  of  classroom  and  laboratory 
course  work  and  then  is  assigned  an  AIC  mentor.  For  the  next  12  weeks,  the  resident 
and  the  mentor  team  work  on  the  application.  The  resident,  now  a  graduate  from  the 
AIC’s  residency  program,  returns  to  the  product  division  with  a  completed  project. 


A  13 


REFERENCES 


[A-  1]  Barker,  Virginia  E.  and  Dennis  E.  O’Connor.  "Expert  Systems  for 
Configuration  at  Digital:  XCON  and  Beyond."  Communications  of  the 
ACM,  Vol  32,  No.  3  (Mar  1989). 

[A-  2]  Telephone  conversation  with  Jack  Rahaim,  Corporate  Artificial 
Intelligence  Marketing  Manager,  and  Themis  Papageorge,  Manager, 
Strategic  Planning  and  Third  Party  Programs,  Digital  Equipment 
Corporation.  1989. 

[A-  3]  Feigenbaum,  P.  E.,  McCorduck,  and  H.P.  Nii.  The  Rise  of  the  Expert 
Company.  Times  Books,  1988. 

[A-  4]  McDermott,  John.  "Rl:  The  Formative  Years,”  A1  Magazine,  Summer 
1981. 

[A-  5]  "Digital  Equipment  Corp.  Saves  Millions  with  Expert  Systems!”  Expert 
Systems  Strategies,  3:  8.  Harmon,  Paul,  ed.  Cutter  Information  Corp. 
1987. 

[A-  6]  Lynch,  Frank,  Charles  Marshall,  Dennis  O'Connor,  and  Mike  Kiskiel,  n. 
"AI  in  Manufacturing  at  Digital.”  The  AI  Magazine,  Winter  1986. 

[A-  7]  Telephone  conversation  with  Bruce  Martin,  Program  Manager  for  Expert 
Systems,  Artificial  Intelligence  Project  Office,  International  Business 
Machines  Corp.,  Jun  1989. 

[A'  8]  Telephone  conversation  with  Candice  Matteucci,  Staff  Systems  Analyst, 
Artificial  Intelligence  Support  Center,  International  Business  Machines 
Corporation.  Jun  1989. 

[A-  9]  The  Boeing  Company  Annual  Report,  1988,  pp.  4  — 22. 

[A-  lOJ  Telephone  conversation  with  Steve  Yanagimachi,  Project  Manager,  Center 
for  Artificial  Intelligence,  Boeing  Corp.,  Jun  1989. 

[A- 11]  Telephone  conversation  with  Perry  Thorndyke,  Strategic  Applications 
Program  Manager,  FMC  Corp.,  Jun  1989. 

[A-12]  Williamson,  Mickey.  "At  Du  Pont,  Expert  Systems  Are  Key  to  AI 
Implementation,”  PC  Week,  13  Jan  1987. 


A  14 


[A-13]  Telephone  conversation  with  Edward  Mahler,  AI  Program  Manage 
DuPont  Co.,  Jun  1989. 


APPENDIX  B 


DoD  ARTIFICIAL  INTELLIGENCE  TRAINING 


In  order  to  establish  artificial  intelligence  (AI)  as  an  effective  logistics  tool  able 
to  attack  and  solve  new  and  difficult  management  problems,  AI  must  be  widely 
understood  as  a  maturing  technology  that  is  ready  for  use  today.  To  establish  that 
recognition,  personnel  at  decision-making  levels  must  become  aware  of  its 
capabilities.  Training  programs  must  be  developed  for  personnel  responsible  for 
daily  operations  and  also  for  all  levels  of  management  personnel. 

The  significant  policy  issues  include  whether  a  specialized  AI  logistics  training 
program  should  be  established  and,  if  so,  what  it  should  look  like  and  who  should 
manage  it. 

THE  REQUIREMENT  FOR  A  SPECIALIZED 
AI  TRAINING  PROGRAM  FOR  LOGISTICS 

If  we  examine  the  history  of  conventional  automatic  data  processing  (ADP) 
training,  we  can  draw  parallels  to  AI  training  and  develop  a  more  comprehensive  and 
robust  DoD  AI  training  program.  Traditionally,  ADP  training  programs  have 
concentrated  on  the  technical  aspects  of  designing  and  implementing  effective 
software  systems.  Most  ADP  training  programs  have  not  tried  to  impart  any 
knowledge  of  the  domain  areas  for  which  the  software  systems  were  intended.  For 
example,  very  few  software  designers  have  been  trained  in  the  functions  and 
operations  necessary  for  a  successful  logistics  program.  Conversely,  logistics 
training  has  traditionally  ignored  the  complexiti^-s  of  designing  and  implementing 
the  softv/are  systems  necessary  for  effective  logistics  support.  These  training  flaws 
have  led  to  a  weakness  in  the  logistician’s  ability  to  convey  system  requirements  to 
ADP  developers  and  to  a  similar  weakness  in  the  ADP  developer’s  ability  to 
understand  the  intricacies  and  subtlety  involved  in  managing  and  implementing  an 
effective  logistics  program.  This  difficulty  in  communication  between  ADP 
developers  and  logisticians  manifests  itself  in  poor  data  system  design,  unnecessary 
schedule  slippages,  and  cost  overruns  caused  by  frequent  changes  in  requirement 
specifications. 


If  we  follow  the  same  training  approach  with  AI,  we  can  expect  to  see  even 
greater  problems.  The  complexity  of  the  task  the  software  must  perform  makes 
developing  requirements  specifications  for  conventional  ADP  software  difficult  even 
when  the  communication  between  the  logistician  and  the  system  is  good.  The  tasks 
AJ  systems  are  required  to  perform  are  generally  an  order  of  magnitude  more 
complex  than  the  tasks  performed  by  conventional  ADP  software. 

If  DoD  tries  to  implement  AJ  systems  on  a  large  scale  using  the  traditional  ADP 
approach  to  training,  it  will  compound  an  already  complex  problem  by  interjecting  a 
communication  barrier  between  logisticians  and  AI  software  developers.  The 
solution  to  this  problem  is  to  ensure  logisticians  at  all  levels  understano  the 
complexities  ^f  AI  software  design  and  implementation  and  that  AI  software 
developers  have  an  understanding  of  the  logistics  process. 

Since  AJ  will  be  used  at  all  levels  of  decision  making,  the  personnel  responsible 
for  making  decisions  at  strategic,  tactical,  and  organizational  levels  must  be  trained 
to  understand  what  problems  are  appropriate  for  AI  solutions,  the  capabilities  and 
limitations  of  .‘J  technology,  and  what  is  involved  in  the  process  of  designing  and 
implementing  AJ  software.  All  logisticians  certainly  do  not  need  to  know  how  to 
perform  knowledge  engineering  or  how  to  design  and  implement  AJ  software;  rather, 
all  should  understand  the  basics  of  the  process.  A  major  component  of  expert  system 
software  development  that  every  logistician  should  understand  is  knowledge 
acquisition.  Knowledge  acquisition  deals  with  obtaining  and  understanding  domain- 
specific  problem  solution  technicues.  Learning  knowledge-acquisition  techniques 
teaches  logisticians  how  to  logically  analyze  logistics  problems  and  enables  them  to 
prepare  more  effective  specifications  for  automated  solutions  to  their  problems. 

Expert  systems  is  the  AJ  technology  most  intimately  connected  with  the  domain 
iii  which  it  is  applied.  Therefore,  it  is  most  important  that  domain  practitioners  have 
a  good  understanding  of  what  is  involved  in  developing  expert  systems  and  what  the 
capabilities  and  limitations  of  those  systems  are.  Other  fields  of  AJ  are  not  as  closely 
tied  to  .specific  domain  knowledge  as  expert  systems;  therefore,  logisticians  do  not 
have  to  be  as  familiar  with  the  technologies  involved.  However,  it  is  still  important 
that  logisticians  understand  the  capabilities  and  limitations  of  each  of  the  AI 
technologies.  Understanding  the  capabilities  and  limitations  of  AI  technologies  such 
as  vision,  .speech,  and  intelligent  robotics  is  difilcult  since  those  technologies  change 
rapidly,  in  some  cases  almost  monthly.  An  introductory  course  to  AI  can  convey  the 
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capabilities  and  limitations  of  AI  technologies  to  logisticians,  and  within  a  very  short 
time,  the  information  might  be  outdated.  For  example,  when  DoD  Components 
began  Al  training  in  1.936,  natural  Language  technology  was  only  cost-effective  for 
applications  involving  very  large  systems.  At  that  time,  effective  natural  language 
systems  cost  $90,000 —  $100,000.  However,  by  mid-i987,  very  powerful  natural 
language  systems  were  commercially  available  for  less  than  $700.  Managers, 
especially  those  who  will  make  decisions  about  applying  AI  technology,  need  to 
understand  that  the  information  provided  in  AI  training  is  changing  constantly,  and 
that  systems  that  may  not  have  been  feasible  at  the  time  of  their  training  may  be 
excellent  applications  6  months  later.  Managers  should  attend  annual  refresher 
courses  in  AI  technology  or  take  other  active  measures  to  stay  abreast  of  the 
technology. 

To  summarize,  the  history  of  ADP  training  has  shown  that  logisticians  should 
understand  what  is  involved  in  the  development  of  their  automated  systems, 
including  AI.  Furthermore,  understanding  the  knowledge-acquisition  process  can 
provide  logisticians  valuable  insight  into  their  decision-making  processes.  Finally, 
logisticians  should  understand  the  capabilities  and  limitations  of  other  AI 
technologies  so  that  they  know  what  AI  technology  can  and  cannot  do  for  them. 

A  SAMPLE  DoD  Al  TRAINING  PROGRAM 

Any  training  program  structure  for  an  organization  as  large  as  DoD  must 
recognize  that  differing  levels  of  knowledge  are  needed.  The  need  for  knowledge 
about  AI  varies  not  only  with  the  position  of  an  individual  in  DoD,  but  also  with  the 
likelihood  that  the  'ndividual  will  be  directly  involved  in  an  AI  development  effort. 
While  it  might  be  useful  to  educate,  in  depth,  all  logistics  personnel  at  all  levels  in  AI 
and  e  cpert  systems,  that  approach  is  not  feasible.  More  rational  objectives  are  to 
make  everyone  aware  of  the  technology  and  to  give  the  in-depth  knowledge  to  those 
people  most  likely  to  employ  AI  in  their  jobs. 

The  training  program  outlined  in  this  appendix  recognizes  the  varying  need  for 
knowledge  by  different  people  throughout  DoD.  The  senior  level  courses  are 
intentionally  brief,  since  most  senior  executives  have  little  time  for  training  and  need 
to  acquire  as  much  knowledge  in  as  little  time  as  possible.  The  training  program  also 
recognizes  a  difference  in  the  need  for  knowledge  between  those  who  are  not  likely  tr 
be  involved  in  the  near  future  in  AI  development  and  those  woo  will.  Sample  course 


sequences  discussed  herein  and  the  sample  course  outlines  presented  are  intended  to 
show  the  type  and  extent  of  knowledge  needed  by  different  logisticians  within  DoD. 
The  DoD  AI  Policy  Office  (AIPO)  should  determine  the  specific  training  targets, 
courses,  and  durations. 

Senior  Level  Executives 

A  program  similar  to  Air  Force  System  Commands’  Project  Bold  Stroke  for  Air 
Force  software  should  be  established  for  DoD  logistics  AI.  The  program  would 
educate  general  -'fficers  ar'^  ^„ner  senior  executives  on  the  capabilities  and 
limitations  of  AI  technologies  as  well  as  what  AI  means  to  logistics.  The  more  senior 
decision  makers  know  about  AI  the  better  they  will  be  able  to  make  decisions  about 
its  use  in  their  Services.  Senior  officers  also  need  to  be  educated  on  the  strategic 
implications  of  AI.  AI  can  be  applied  to  strategic  and  national  policy  planning  as  well 
ac  to  the  execution  of  tactical  policies.  A  proposed  AI  training  program  for  general 
officers  and  senior  executives  is  shown  in  Figure  B-1.  (Sample  course  outlines  for  all 
proposed  courses  are  presented  at  the  end  of  this  appendix.) 


^hose  not  directly  involved  m 
AI  system  aevelcpment 

AI  in  logistics  (2  hours) 


Senior  executives 
AI  management  awareness 
(1  hour) 


Problem  selection  (4  hours) 


Knowledge  acquisition  for 
senior  executives 
(1  day) 


Those  directly  involved  m 
AI  system  development 


Introduction  to  AI  and  ES 
for  senior  logistics  managers 
(2  days) 


Note:  55  =  rxpert  Systern. 


FIG.  B-1.  PROPOSED  AI  TRAINING  PROGRAM  FOR  SENIOR  EXECUTIVES 


Middle  Management/First  Level  Supervisors 

Middle  management  and  first  level  supervisory  personnel  need  a  more  thorough 
understanding  of  the  application  of  AI  to  their  daily  operations.  These  managers 
will  ultimately  be  required  to  commit  limited  resources  to  AI  development  efforts. 
Even  if  the  bulk  of  the  AI  development  is  contracted  out,  the  first  levels  of 
management  will  still  be  required  to  commit  their  task  experts  to  the  development 
projects  since  it  will  be  their  experts’  knowledge  that  the  developers  will  capture  and 
place  in  an  automated  system.  Therefore,  it  is  essential  that  personnel  at  this  level 
understand  the  advantages  of  using  AI.  The  role  that  senior  management  should 
play  here  is  to  make  subordinate  management  understand  the  importance  of 
accepting  short-term  costs  in  some  cases  in  order  to  achieve  greater  long-term 
benefits.  As  a  minimum,  all  personnel  at  this  level  should  attend  a  problem 
identification  course  and  a  knowledge  acquisition  course.  This  area  could  see 
tremendous  benefits  from  training  personnel  in  the  knowledge  acquisition  process. 
Conventional  ADP  has  not  been  able  to  deal  effectively  in  the  past  with  the  problems 
of  middle  and  upper  level  m?-agement.  Knowledge  acquisition  will  enable 
management  to  analyze  their  tasks  and  develop  expert  system  requirements  to  better 
solve  these  problems.  Figure  B-2  shows  the  proposed  training  program  for  middle 
managers  and  first-level  supervisors. 


Middle  management/first-leve!  supervisors 
AI  management  awareness  course 
(1  hour) 


Thos8  directly  involved 
Al  system  development 
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AI  system  development 


Problem  selection  (4  hours) 


Introduction  to  AI  and  ES 
for  senior  logistics  managers 
(2  days) 


Knowledge  acquisition 
(2  days) 


FIG.  B-z.  PROPOSED  TRAINING  FOR  MIDDLE  MANAGEMENT 


Line  Personnel 


Line  personnel  will  ultimately  either  use  or  implement  DoD’s  AI  systems  or  will 
deal  with  contractors  who  will  implement  them.  Personnel  in  all  functional  areas 
need  to  understand  what  makes  a  task  a  good  candidate  for  an  AI  application  and 
how  to  properly  scope  and  define  the  application.  Many  personnel  at  this  level  will 
need  to  understand  how  to  go  beyond  simple  problem  identification;  that  is,  they  will 
need  to  know  how  to  develop  and  implement  actual  AI  applications.  As  a  minimum, 
these  personnel  should  attend  a  course  in  problem  selection  and  knowledge 
acquisition.  Personnel  desiring  to  develop  AI  applications  should  also  attend 
additional  courses  in  the  expert  system  (ES)  development  process  and  AI 
development  tool  programming.  Figure  B-3  shows  the  proposed  training  program  for 
line  personnel. 


Line  personnel 

AI  management  awareness  course 
(1  hour) 


Those  not  airectiy  'Hvotved  m 
AI  system  deveiooment 

Problem  selection  (4  hours) 


Those  directly  involved  in 
AI  system  development 

Expert  system  development 
process  (5  days) 


Knowledge  acquisition  (2  days) 


FIG.  B-3.  PROPOSED  TRAINING  PROGRAM  FOR  LINE  PERSONNEL 

MANAGEMENT  OF  THE  AI  TRAINING  PROGRAM 

DoD  has  three  basic  options  on  who  should  manage  AI  training. 

Option  1 :  Each  Service  Could  Manage  Its  Own  Program 

The  Services  nave  already  established  their  training  programs  and  have  met 
with  reasonable  success  to  date.  In  addition,  some  of  the  resources  required  by  this 
option  are  already  in  place.  However,  this  option  represents  a  decentralized  approach 
that  tends  to  diffuse  the  available  AI  training  talent,  which  is  currently  at  a 


premium.  Training  expertise  would  be  dispersed  across  the  Services  and  a  greater 
expenditure  of  personnel  resources  dedicated  to  training  would  be  required.  Hence 
these  individuals  would  not  be  available  to  implement  and  manage  AI  systems. 

Option  2:  OSD  Could  Designate  a  Single  Service 
To  Manage  the  Program  for  DoD 

This  option  has  the  advantage  of  concentrating  AI  training  resources  within 
one  location  and  thus  making  better  use  of  a  scarce  resource.  It  also  has  precedents 
within  the  DoD  software  community.  For  example,  the  Marines  run  the  DoD  OS  360 
Systems  Programming  Course  for  all  Services.  This  option  is  in  accord  with  the  OSD 
preference  for  centralizing  the  procurement  of  AI  hardware,  software,  and  services 
within  one  activity.  Again,  some  of  the  resources  to  accomplish  this  option  are 
already  in  place.  It  makes  sense  to  centralize  the  AI  training  to  use  the  hardware  and 
software  purchased  through  the  central  procurement  source.  The  drawback  to  this 
option  is  that  the  DoD  Components  not  selected  to  manage  the  training  program  may 
be  reluctant  to  participate. 

Option  3:  OSD  Could  Manage  the  Program  Centrally 
Through  the  DoD-Wide  Technology  Transition  Center 

This  option  is  advantageous  in  that  AI  training  resources  would  be 
concentrated  within  one  location.  It  would  offer  the  same  benefits  that  hold  for 
Option  2. 

We  recommend  that  OSD  pursue  Option  3  at  this  time.  This  option  will  be  more 
readily  accepted  by  the  DoD  Components  since  it  will  guarantee  all  equal  access  to  AI 
training.  Central  management  of  training  funds  by  DoD  will  also  ensure  minimal 
duplication  of  effort  when  contracting  for  training  services. 

A  primary  function  of  the  DoD-wide  Technology  Transition  Center  (TTC) 
should  be  to  evaluate  AI  curricula  in  DoD  and  academia  to  determine  their 
suitability  fc  r  use  within  DoD.  After  assessing  the  suitability  of  existing  curricula, 
the  DoD  TTC  should  identify  shortfalls  in  the  existing  curricula  and  establish 
guidelines  fcr  further  incorporation  of  AI  into  logistics  curricula  development  to 
correct  the  existing  shortfalls.  An  additional  function  of  the  DoD  TTC  should  be  to 
establish  minimal  requirements  for  certified  logistics  AI  instructors,  and  those 
requirements  should  be  used  by  the  DoD  Components.  After  the  DoD  TTC  has 
established  the  necessary  curricula  and  minimal  requirements  for  logistics  AI 


instructors,  these  guidelines  could  be  provided  to  "mini-TTCs"  within  the  DoD 
Components.  These  "mini-TTCs”  v/ould  conduct  the  logistics  AI  training. 
Establishment  of  the  mini-TTCs  does  not  preclude  contracting  for  either  the  study  of 
necessary  AI  curricula  or  the  execution  of  the  logistics  AI  training.  The  decision  to 
contract  training  services  or  to  perform  the  work  organically  should  be  at  the 
discretion  of  the  individual  DoD  Component.  The  remainde*'  of  this  appendix 
presents  sample  AI  training  course  outlines. 


SAMPLE  DoD  AI  TRAINING  COURSE  OUTLINES 


Management  Awareness  of  AI 
(1-hour  course) 

I.  Introduction/Overview 
n.  Artificial  Intelligence  Technologies 

A.  Expert  systems 

B.  Natural  language 

C.  Speech 

D.  Vision 

E.  Intelligent  robotics 

F.  Neural  networks 

HI.  State  of  the  Art  in  Applied  AI 

A.  Capabilities/success  stories 

B.  Limitations/failures 
rV.  Applications  in  Logistics 
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AI  in  Logistics 
(2-hour  course) 

I.  Introduction/Overview 
n.  Basics  of  AI  &  Expert  Systems  for  Managers 
in.  Benefits  and  Risks 
rV.  Logistics  Applications  of  AI 

A.  Transportation 

B.  Inventory  management 

C.  Maintenance 

D.  Warehouse  management 

E.  Integrated  logistics  support 

F.  Acquisition 

V.  Strategies  for  Implementing  AI  Programs 
VT.  Other  Advanced  Technologies  and  Logistics 
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Expert  Systems  Problem  Selection 
(4-hour  course) 


I.  Introduction/Overview 
n.  Applying  Expert  Systems 
in.  Limitations  of  Expert  Systems 
rV''.  Problem  Selection  Criteria 

A.  Is  an  expert  system  possible? 

B.  Is  an  expert  system  justified? 

C.  Is  an  expert  system  appropriate? 

V.  Problem  Selection  Exercises  and  Case  Studies 
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Introduction  to  AI  and  Expert  Systems  for  Senior  Logistics  Managers 

(2-day  course) 

I.  Introduction/Overview 
n.  AI  Technologies 

A.  Expert  systems 

B.  Natural  language 

C.  Speech  recognition 

D.  Vision 

E.  Intelligent  robotics 

F.  Neural  networks 

in.  Applying  Expert  Systems 

IV.  Limitations  of  Expert  Systems 

V.  Problem  Selection  Criteria 

A.  Is  an  expert  system  possible? 

B.  Is  an  expert  system  justified? 

C.  Is  an  expert  system  appropriate? 

VI.  Case  Studies 

V'n.  Development  Resources  Required 
Vm.  Expert  System  Life  Cycle 
IX.  Testing  Expert  Systems 
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Expert  System  Development  Process 
(5-d.ay  course) 

I.  Knowledge-Based  System  Life  Cycle 

A.  Requirements  analysis 

1.  Knowledge  to  be  acquired  during  the  initial  phase 

2.  Type  and  scope  of  the  problem 

3.  Expertise  needec. 

4.  Time  needed  to  develop  the  system 

5.  Resources 

6.  Goals 

7.  Estimating  return  on  investment 

B.  Preliminary  design 

1.  Knowledge  to  be  acquired  during  later  phases 

2.  Gathering  concepts,  relations,  and  control  mechanisms 

3.  Identifying  subtasks,  strategies,  and  constraints 

4.  Level  of  detail  needed  for  the  knowledge  base 

C.  Detailed  design 

1.  Selecting  a  tool/shell 

2.  Developing  a  knowledge-base  structure  and  representations 

3.  Planning  the  delivery  environment  and  user  interface 

D.  Code  and  unit  test 

1.  Coding  the  knowledge  base 

2.  Testing  the  knowledge  base 

E.  System/integration  testing 

1.  Verification  and  validation 

2.  Performance  testing  (dynamic  testing) 

(a)  Evaluating  the  performance  of  knowledge-based  system 
fb)  Evaluating  the  accuracy 
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Expert  System  Development  Process  (Continued) 

3,  Structural  testing  (static  testing) 

(a)  Consistency 

(b)  Completeness 
F.  Maintenance 

1.  Correcting  deficiencies 

2.  Improving  performance 

3.  Improving  accuracy 

4.  Enhancing  usability 

5.  Microcosm  of  development  process 
n.  Resources 

A.  Team  structure 

B.  Schedule 

C.  Documentation 

1.  Types  of  documentation 

2.  Levels  of  documentation 

D.  Cases 

1.  Kinds  of  cases 

2.  Number  of  cases 

3.  Case  management  facilities 
ni.  Interviewing  Techniques 

A.  Methodologies 

B.  Dealing  with  difficult  experts 

C.  Combining  knowledge  from  multiple  experts 
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Knowledge  Acquisition  for  Senior  Executives 
(1-day  course) 

I.  Introduction 
n.  Requirements  Analysis 

A.  Knowledge  to  be  acquired  during  initial  phase 

B.  Type  and  scope  of  the  problem 

C.  Expertise  needed 

D.  Time  needed  to  develop  the  system 

E.  Resources 

F.  Gcals 

G.  Estimating  return  on  investment 
m.  Preliminary  Design 

A.  Knowledge  to  be  acquired  during  later  phases 

B.  Gathering  concepts,  relations,  and  control  mechanisms 

C.  Identifying  subtasks,  strategies,  and  constraints 

D.  Level  of  detail  needed  for  the  knowledge  base 
rV.  Interviewing  Techniques 

A.  Methodologies 

B.  Dealing  with  difficult  experts 

C.  Combining  knowledge  from  multiple  experts 


Knowledge  Acquisition 
(2-day  course) 

I.  Introduction 
n.  Requirements  Analysis 

A.  Knowledge  to  be  acquired  during  initial  phase 

B.  Type  and  scope  of  the  problem 

C.  Expertise  needed 

D.  Time  needed  to  develop  the  system 

E.  Resources 

F.  Goals 

G.  Estimating  return  on  investment 
ni.  Preliminary  Design 

A.  Knowledge  to  be  acquired  during  later  phases 

B.  Gathering  concepts,  relations,  and  control  mechanisms 

C.  Identifying  subtasks,  strategies,  and  constraints 

D.  Level  of  detail  needed  for  the  knowledge  base 
rV.  Detailed  Design 

A.  Selecting  a  tool/shell 

B.  Developing  a  knowledge-based  structure  and  representations 

C.  Planning  the  delivery  environment  and  user  interface 
V.  Interviewing  Techniques 

A.  Methodologies 

B.  Dealing  with  difficult  experts 

C.  Combining  knowledge  from  multiple  experts 
VI.  Knowledge  Acquisition  Exercises 
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APPENDIX  C 


KEY  TARGET  APPLICATIONS  OF  ARTIFICIAL  INTELLIGENCE 

TO  LOGISTICS 


The  key  logistics  applications  of  artificial  intelligence  (AI)  are  presented  here  >n 
terms  of  near-term  and  long-term  initiatives.  Near-term  initiatives  are  those  that 
rely  on  existing,  mature  technology  and  can  be  implemented  without  further 
research  and  development.  Long-term  initiatives  involve  relatively  mature 
laboratory  technologies,  but  they  are  technologies  that  have  not  yet  been  applied  in  a 
real-world  environment  and  may,  therefore,  require  additional  research  and 
development. 

This  list  of  key  target  applications  is  not  intended  to  be  all  inclusive.  In  fact, 
one  of  the  early  tasks  of  the  OSD  Logistics  and  AI  Policy  Council  should  be  to 
determine  the  specific  target  applications  for  DoD  and  the  emphasis  to  be  given  to 
each.  One  of  the  key  findings  in  the  review  of  corporate  involvement  with  AI  was  the 
necessity  to  examine  AI  from  the  perspective  of  the  essential  business  needs  of  the 
organization.  Table  C-1  provides  a  framework  OSD  can  use  in  comparing  the  AI 
technologies  with  the  organizational  needs.  Obviously,  the  list  of  organizational 
needs  included  in  Table  C- 1  is  only  a  beginning  of  the  actual  list  of  OSD’s  needs. 

The  number  of  potential  high-leverage  applications  in  DoD  logistics  is 
enormous.  The  productivity  and  qo.ality  gains  to  be  realized  from  these  potential 
applications  are  so  significant  that  DoD  should  place  special  emphasis  on  the 
development  of  software  to  meet  the  needs  of  key  DoD  Components  responsible  for 
these  functions. 

NEAR-TERM  APPLICATIONS  (1  TO  3  YEARS) 

Off-Equipment  Maintenance 

We  recommend  that  OSD  establish  a  DoD- wide  policy  on  Al-based  maintenance 
systems  for  newly  acquired  weapon  systems  and  nonweapon  capital  equipment. 
Given  the  current  state  of  technology,  every  weapon  system  procured  by  the  Services 
should  be  delivered  with  intelligent  maintenance  aids.  As  part  of  the  weapon 
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system’s  support  equipment,  defense  contractors  should  be  required  to  deliver 
knowledge-based  systems  (KBS)  capable  of  diagnosing  and  isolating  faults  to  a  level 
sufficient  for  both  organizational  and  depot-level  maintenance  functions.  The 
organizational-level  KBS  could  easily  be  required  to  fit  inside  a  computer  system  no 
larger  than  a  lap-top  computer  and  in  some  cases,  a  hand-held  computer.  The 
depoc-level  KBS  must  be  integrated  into  the  designated  automatic  test  equipment. 
Diagnostic  applications  such  as  these  will  reduce  the  number  of  personnel  required  to 
maintain  systems,  reduce  the  number  of  Cannot  Duplicates  and  Retest  Okays,  reduce 
the  time  required  to  isolate  and  repair  components,  and  reduce  the  number  of  spares 
neede*^  *’o  support  the  weapon  systems. 
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Transport  Loading  (Ship,  Plane,  and  Truck) 


A  major  problem  confronting  military  logisticians  today  is  how  to  efficiently 
load  cargo  transports  so  that  the  vital  material  is  accessible  and  unloading  time  is 
minimized.  Efficient  loading  is  especially  critical  iu  combat  situations  in  which 
certain  materials  must  be  unloaded  as  rapidly  as  possible.  In  setting  up  forward 
bases,  some  materials  may  need  to  be  unloaded  and  set  up  before  other  materials  can 
be  unloaded.  When  the  complexities  of  proper  weight  and  balance  are  taken  into 
account  for  aircraft  and  ships,  the  problem  of  properly  loading  a  transport  becomes 
even  more  complex.  Both  the  Air  Force  and  the  Navy  are  making  some  early 
attempts  to  develop  this  kind  of  KBS,  duo  iheir  efforts  need  to  be  expanueu  so  that  all 
military  airports  and  shipping  terminals  have  access  to  the  technology. 

Route  Planning 

In  periods  of  hostility,  rapid  and  accurate  route  planning  may  be  crucial  to  the 
success  of  logistics  operations.  Systems  need  to  be  developed  that  consider  route 
time,  distance,  terrain  and  meteorological  conditions,  and  potential  military  threats 
in  the  area.  Many  route-planning  systems  are  being  developed  for  commercial 
applications  and  even  a  few  for  military  logistics.  However,  few  if  any  consider 
factors  other  than  time  and  distance.  While  minimizing  time  and  distance  is 
certainly  important,  military  logisticians  must  also  take  into  account  intervening 
terrain,  meteorological  conditions,  fuel  availability,  equipment  capabilities,  threats 
from  the  enemy,  and  time-critical  situations.  These  tools  could  also  serve  as  valuable 
training  aids  and  allow  war  planners  to  simulate  various  wartime  scenarios. 

Data  Validation 

While  da  ri  v  -'idation  may  not  seem  as  important  as  applications  such  as  fault 
diagnosis  and  ro..  .j  planning,  its  value  is  obvious  from  the  tremendous  amount  of 
press  generated  when  someone  sells  DoD  $400  hammers.  Data  validation  systems 
can  play  important  roles  as  intelligent  front  ends  to  DoD’s  purchasing  and 
requirements  databases.  These  systems  in  effect  extend  the  editing  capabilities  of 
the  existing  conventional  data  systems.  Today’s  data  systems  perform  exceedingly 
well  at  detecting  and  preventing  incorrect  types  of  data  from  being  entered  by 
humans.  Errors  such  as  entering  "$lw3.00”  instead  of  "$123.00”  are  detected. 
Conventional  data  systems  are  not  capable  of  detecting  errors  where  the  data  type  is 
correct  but  the  value  is  inaccurate,  errors  such  as  "$132.00”  when  "$123.00”  was 


intended.  Such  errors  are  also  the  most  frennent  type  of  data  entry  error.  They  arise 
out  of  misplaced  decimal  points,  transposed  digits,  or  simply  hitting  the  wrong  key  on 
the  input  data  terminal.  Simple  data  range  tests  are  frequently  incapable  of 
handling  such  subtle  errors.  KBS  technology  allows  system  designers  to  develop  the 
capability  of  detecting  inaccurate  data  values  by  applying  human-like  reasoning  to 
the  data  entry  process.  These  applications  can  save  millions  of  dollars. 

Spares  Determination 

Many  conventional  modeling  techniques  have  been  developed  to  determine 
when  and  where  spares  will  be  needed  in  the  event  of  hostilities.  However,  these 
models  take  a  fairly  static  view  of  the  world  and  can  be  difficult  to  use.  Several  hours 
of  work  may  be  required  to  determine  what  parameters  need  to  be  entered  into  the 
system  to  obtain  a  reasonable  response.  Further,  these  models  generally  assume 
fairly  constant  attrition  rates  of  men  and  material  and  make  no  allowances  for 
cannibalization  of  equipment.  These  limitations  are  primarily  due  to  the 
combinatorics  of  the  situation;  if  these  systems  considered  all  possible  variables,  they 
would  take  far  longer  to  run  and  would  have  only  limited  utility.  Intelligent 
simulation,  i.e.,  KBS  combined  with  conventional  modeling  techniques,  can  make 
these  systems  easier  to  use  and  much  more  flexible,  while  increasing  the 
responsiveness  of  the  models. 

Acquisition  Support 

Writing  and  assembling  procurement  packages  has  become  a  tedious  and 
complex  problem.  Still,  the  majority  of  DoD’s  contracting  expertise  resides  in-house. 
KBS  offer  the  opportunity  to  capture  DoD  procurement  expertise  and  assemble  it  in  a 
system  capable  of  preparing  complete  procurement  packages. 

Several  small  systems  attempt  to  solve  one  small  part  of  this  very  large 
problem.  The  Software  Support  Decision  Aid  identifies  the  correct  DoD-STD-2l67A 
data  item  descriptions  (DIDs)  to  be  placed  on  contract  to  procure  software 
documentation.  The  Expert  Tailoring  Assistant  Design  for  Testability  develops 
statement  of  work  (SOW)  tasks  for  including  system  testability  in  system  design. 
Systems  need  to  be  developed  for  all  aspects  of  weapon  system  procurement  and  then 
integrated  into  a  single  acquisition  support  system. 
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Indeed,  the  major  finding  of  a  1988  study  on  the  current  status  of  automated 
SOW  systems  by  DoD  organizations  was  that  "SOW  automation  system  development 
in  DoD  is  proceeding  in  a  fragmented,  uncoordinated  fashion.  Some  degree  of 
coordination  and  control  appears  desirable.”  [C-1]  Another  suggestion  of  the  study 
was  that  Computer-Aided  Acquisition  Logistic  Support  (CALS)  requirements  be  built 
into  the  system.  The  requirement  that  a  contractor  provide  a  knowledge-based 
diagnostic  aid  with  the  prime  weapon  system  could  also  be  a  built-in  requirement. 
Another  feature  recommended  for  automated  systems  is  a  schedule  consistency 
checker. 

While  thi3  was  not  a  point  of  concentration  in  this  study,  inconsistencies 
between  related  data  elements  is  a  very  common  problem  in  practice.  T’he 
engineering  drawing.  Reliability  and  .Maintainability  (R&.M),  Logistics 
Support  Analysis  Record  (LSAR),  and  Integrated  Logistics  Support  (ILS) 
element  Data  Item  Descriptions  (DIDs)  are  an  outstanding  example.  The 
problem  occurs  because  DoD  schedules  backwards,  i.e.,  based  on  data  needs 
to  meet  deployment  and  other  higher  level  schedule  needs,  without  regard  to 
when  the  data  can  be  logically  available.  .A  schedule  consistency  check  can 
be  built  into  a  Contract  Data  Requirements  List  (CDRD/Data  Element 
Selection  automatec  system  at  reasonable  cost. 

The  advantages  of  a  set  of  KBS  for  acquisition  support  could  be  tremendous, 
especially  if  implemented  DoD-wide.  Federal  acquisition  regulations  permit  a  great 
deal  of  technical  variation  around  a  core  of  basic  requirements.  A  benefit  of 
acquisition-oriented  systems  would  be  their  ability  to  facilitate  consistent 
performance  in  developing  boilerplates  and  the  completion  of  frequent,  repetitive 
procurements  of  noncomplex  items  or  services.  Contracts  would  be  more  complete 
since  the  KBS  would  ensure  important  aspects  of  the  contract  were  not  overlooked. 
Finally,  the  KBS  could  keep  an  audit  trail  of  the  decisions  made  in  the  development 
of  the  procurement  package. 

Integrated  Logistics  Support 

Integrated  Logistics  Support  consists  of  many  interrelated  and  independent 
logistics  areas  that  must  be  examined  in  great  detail  when  developing  and 
supporting  weapon  systems.  Very  seldom  do  acquisition  and  support  organizations 
have  personnel  knowledgeable  in  all  areas  of  ILS.  Yet  the  organizations  must 
develop  requirements  and  documents  such  as  the  Integrated  Logistics  Support  Plan 
lILSP).  the  Computer  Resources  Life  Cycle  Management  Plan  (CRLCMP),  and 
others.  KBS  now  allow  DoD  to  capture  expertise  in  each  of  the  ILS  elements  and 
provide  that  expertise  in  electronic  form  to  acquisition  and  support  organizations. 
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That  ability  will  be  of  great  value  to  OSD  in  the  near  term.  For  example,  a  near-term 
system  might  guide  a  logistician  through  the  review  process  for  a  specific  element  of 
ILS,  such  as  the  manning  required  to  support  a  new  weapon  system. 

Nondestructive  Inspection 

Intelligent  Nondestructive  Inspection  (NDI)  systems  with  the  capability  to 
automatically  analyze  component  X-rays,  spectrograms,  eddy  current  patterns,  and 
fluoropenetrant  dye  patterns  will  greatly  reduce  the  time  it  takes  to  analyze 
mechanical  structures.  Expert  systems  have  been  analyzing  spectrograms  for  years. 
For  example,  DENDRAL  was  an  expert  system  designed  to  decipher  molecular 
structures  based  on  the  spectrogram  of  chemical  compounds. 

Currently,  the  major  impediment  to  application  of  AI  to  NDI  is  the  lack  of  an 
adequate  vision  system.  However,  the  type  of  vision  necessary  for  interpretation  of 
images  from  X-ray  photography  and  fluoropenetrant  dyes  is  an  order  of  magnitude 
simpler  than  that  required  for  three-dimensional  scene  recognition.  Hence,  these 
systems  should  oe  available  within  the  next  2  to  3  years.  General  Electric  is 
currently  working  at  applying  neural  networks  to  interpreting  both  fluoropenetrant 
dye  and  X-ray  images.  GE  expects  to  have  a  system  for  fluoropenetrant  dye 
inspection  available  within  the  next  year.  These  systems  will  increase  the  accuracy 
with  which  defects  in  mechanical  structures  are  identified.  In  the  long  term,  when 
combined  with  flexible  machining  centers,  NDI  systems  will  automatically  repair  the 
defects.  Other  long-term  applications  are  addressed  in  the  following  section. 

LONG-TERM  APPLICATIONS  (3  TO  5  YEARS) 

Equipment  Decontamination 

Chemical,  biological,  and  radiological  (CBR)  environments  are  highly 
hazardous  to  humans.  The  combination  of  KBS,  three-dimensional  vision,  and 
advanced  robotics  will  soon  make  possible  the  development  of  intelligent  robots 
capable  of  operating  in  the  real  world  and  performing  complex,  hazardous  tasks.  An 
equipment  decontamination  system  could  operate  in  the  field  to  autonomously 
decontaminate  ships,  planes,  tanks,  and  any  other  contaminated  vehicles.  These 
systems  would  eliminate  the  need  for  support  personnel  to  enter  hazardous  CBR 


environments. 


On-Equipment  Maintenance 

As  embedded  computer  systems  become  faster  and  more  powerful,  off- 
equipment  diagnostic  systems  can  be  embedded  into  weapon  systems.  They  will 
provide  the  same  benefits  previously  discussed  with  the  added  benefit  of  increased 
mobility  and  reduced  support  equipment. 

Damage  Assessment 

Ships,  planes,  ground  vehicles,  and  facilities  are  subject  to  severe  battle  damage 
that  may  render  their  on-board  diagnostic  systems  inoperative.  Battle  damage 
assessment  systems  will  provide  the  capability  to  identify  damaged  components  and 
recommend  repair  actions  to  maintenance  personnel  based  on  external  visual 
inspection  of  the  vehicle  or  facility.  Eventually,  these  systems  may  be  able  to  make 
the  repairs  themselves. 

Automated  Repair  Systems 

In  the  longer  term,  many  of  the  diagnostic  systems  developed  in  the  next 
1  to  3  years  will  be  expanded  to  automated  repair  systems.  They  will  accept  failed 
electronic  components,  determine  the  source  or  sources  of  failures,  and  execute  the 
repair  procedures.  These  systems  will  be  able  to  deal  with  multiple  failures  within 
single  components. 

Flexible  Machining  Centers 

Flexible  machining  centers  will  combine  intelligent  NDI  with  computer- 
controlled  manufacturing  and  repair  techniques  to  automatically  produce  good 
mechanical  parts.  These  systems  will  operate  much  like  the  automated  repair 
systems.  Flexible  machining  centers  will  accept  failed  components  such  as  jet  engine 
disks  or  tank  armor,  apply  the  appropriate  NDI  procedure  to  detect  the  structural 
flaws,  and  determine  the  most  appropriate  repair  process.  The  system  will  then  carry 
out  the  repair  process,  apply  the  necessary  quality  checks,  and  produce  a  repaired 
component. 

RESEARCH  WILL  BE  REQUIRED 

The  applications  cited  herein  should  be  followed  up  with  research.  However, 
the  first  step  in  this  process  is  for  OSD  to  determine  its  specific  needs  and  then  match 
those  needs  against  existing  technical  capabilities  for  each  AI  technology  area.  With 


that  analytic  approach,  OSD  can  determine  the  major  areas  in  which  research  is 
required  to  support  the  logistics  AI  initiatives.  Research  will  be  required  in  several 
areas  and  should  be  supported  and  funded.  Major  policy  issues  to  be  resolved  by  the 
OSD  Logistics  and  AI  Policy  Council  concern  (1)  how  these  research  initiatives  will 
be  funded  and  (2)  the  process  for  identifying  the  research  requirements  and  setting 
their  priorities. 
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DETERMINING  WHEN  TO  INVEST  IN  ARTIFICIAL  INTELLIGENCE 


This  appendix  provides  criteria  and  other  relevant  information  for  the  selection 
and  justification  of  artificial  intelligence  (AJ)  projects.  It  ofYers  useful  guidelines  for 
the  Military  Services  in  making  their  selections. 

SCREENING  ARTIFICIAL  INTELLIGENCE  PROJECTS 

Since  the  criteria  forjudging  the  value  of  nominated  AJ  projects  vary  to  some 
extent  with  the  technology  being  considered,  we  havp  divided  this  section  along  AI 
technology  lines.  We  discuss  in  order  the  criteria  for  judging  the  value  of  expert 
knowledge-based  systems  (KBS)  applications,  natural  language  systems,  speech 
recognition/ understanding  systems,  three-dimensional  vision  systems,  intelligent 
robotic  systems,  and  neural  networks. 

Three  major  factors  must  be  considered  when  evaluating  AI  projects:  the 
feasibility,  appropriateness,  and  economic  justification  for  the  project.  First,  one 
must  determine  whether  an  AI  solution  is  feasible,  i.e.,  whether  the  success  of  the 
project  in  achieving  its  goals  through  the  application  of  AI  technology  is  reasonably 
probable.  The  second  factor  is  the  appropriateness  of  applying  an  AI  solution,  i.e.. 
whether  the  use  of  AI  techniques  is  the  best  solution  or  whether  more  conventional 
techniques  would  serve  as  well.  The  final  factor  that  must  be  considered  is  the 
economic  justification  or  payback.  Will  the  solution  be  justified?  What  is  the  value 
added  to  the  mission  efTectiveness  of  the  using  organization? 

In  the  following  sections,  we  discuss  the  criteria  for  judging  the  feasibility, 
appropriateness,  and  justification  of  developing  an  AI  application.  We  examine  a 
payback  model  and  the  merits  of  standardized  payback  models.  Finally,  we  propose 
criteria  to  establish  priorities  for  nominated  AI  projects. 

Expert/Knowledge-Based  Systems 

An  experVKBS  requires  several  features.  First,  at  least  one  human  expert  or 
comparable  source  of  knowledge  must  be  available  so  that  the  system  developer 


(knowledge  engineer!  can  extract  knowledge  about  the  task  to  be  performed. 
Further,  managem  „t  must  be  willing  to  commit  a  substantial  amount  of  the  expert’s 
time  to  the  des.'  jpment  project. 

The  expert  must  be  capable  of  communi'-ating  knowledge  about  the  task  to  the 
knowledge  engineer  with  enthusiasm.  The  task  must  require  that  the  expert  exercise 
some  judgmental  knowledge.  The  expert  does  not  have  to  be  physically  present  to 
perform  the  task;  the  system  developer  and  the  expert  should  be  able  to  communicate 
telephonically,  and  the  system  developer  should  be  able  to  perform  the  task  by  simply 
receiving  verbal  instructions  from  the  expert. 

The  task  should  not  require  the  use  of  common  sense.  Common  sense  is  the 
knowledge  that  most  humans  draw  on  to  be  able  to  function  in  the  real  world.  The 
knowledge  is  extremely  diverse  and  requires  access  to  a  vast  storehous'^  of 
information.  Encoding  of  common  sense  knowledge  is  still  beyond  the  capabilities  of 
AI  systems. 

A  task  is  appropriate  when  certain  characteristics  indicate  that  the  best 
approach  is  an  expertKBS  and  when  more  conventional  solutions  are  not  sufficient. 
The  task  should  be  defined  clearly  and  should  require  the  use  of  heuristics  or  rules-of- 
thumb.  It  may  require  that  a  solution  be  derived  in  the  absence  of  complete  or  certain 
Knowledge  At  the  start  of  the  project,  the  knowledge  engineer,  the  expert,  and  the 
users  must  have  a  clear  understanding  of  exactly  what  the  KBS  will  do. 

Given  today’.s  level  of  technological  development,  the  task  should  be  one  that 
can  be  performed  by  the  human  expert  in  12  to  S  nours.  If  the  task  can  be  performed 
too  quickly  by  the  expert  ( i.e.,  in  less  than  half  an  hour)  then  it  is  probably  one  that  is 
ton  simple  to  warrant  developing  an  expert  KBS.  On  the  other  hand,  if  the  task  takes 
the  expert  more  than  8  hours  to  perform,  it  is  probably  too  complex  to  be  developed 
within  a  reasonable  amount  of  time. 

The  task  should  be  sufliciently  narrow.  The  expert  KBS  should  not  deal  with  a 
wide  range  of  topics;  it  should  be  confined  to  one  specific  task.  If  the  task  is  not 
narrowly  defined,  the  KBS  will  tend  to  become  too  large  to  be  manageable  and  the 
development  team  will  have  a  greater  tendency  to  .stray  from  the  original  purpose  of 
the  development  elTort. 


If  a  conventional  computer  program  can  perform  the  task  with  an  acceptable 
degree  of  precision  within  an  acceptable  period  of  time,  the  most  appropriate  solution 
is  probably  the  conventional  computer  program.  Such  a  choice  does  not  mean  that  a 
KBS  could  not  perform  the  task  as  well  as  the  conventional  solution;  it  means  that 
conventional  solutions  are  generally  more  readily  accepted  because  they  do  not 
require  special  expertise  to  maintain  or  support  and  easily  fit  with  other 
conventional  systems. 

Performance  of  the  cask  should  require  the  manipulation  of  symbolic 
information,  or  information  that  is  represented  by  such  woras  as  large,  small,  blue,  or 
green.  Symbolic  information  has  no  intrinsic  meaning  or  value  in  itself;  its  'inly 
meaning  or  value  is  what  we  infer.  It  may.  at  times,  be  ambiguous.  N umenc  data.  ■  m 
the  other  hand,  is  unambiguous  and  is  readily  processed  by  conventional  computer 
programs.  If  the  task  can  be  performed  solely  by  manipulating  numeric  information, 
then  it  can  probably  best  be  performed  using  a  conventional  computer  program. 
However,  a  valid  application  for  a  KBS  that  deals  with  numeric  data  could  be  the 
determination  of  the  correct  numbers  to  be  used  in  an  algorithm  under  different  sets 
of  circumstances.  In  those  situations,  the  numeric  information  is  part  of  the 
conclusion.  The  description  of  the  situation  that  determines  which  numeric  data  to 
use  is  the  symbolic  information  the  system  will  process. 

Natural  Language  Processing 

Two  factors  should  be  considered  when  determining  the  feasibility  of  applying  a 
natural  language  system.  The  first  concern  is  the  size  of  the  vocabulary  needed  to 
perform  the  task.  As  a  general  rule,  the  vocabulary  needed  to  perform  the  task 
should  not  be  larger  than  a  few  hundred  words.  If  more  than  a  few  hundred  words  are 
required,  the  system  will  not  perform  as  rapidly  and  the  ambiguities  in  the 
vocabulary  will  be  difficult  to  resolve.  The  complexity  of  the  grammar  needed  to 
convey  the  concepts  about  the  tasks  being  performed  must  also  be  considered. 
Current  state-of-the-art  natural  language  systems  are  unable  to  deal  with  very 
complex  sentences.  Therefore,  the  concepts  about  tasks  to  be  performed  by  natural 
language  systems  should  be  ones  that  can  be  stated  in  sentences  composed  of  phrases 
connected  by  simple  Boolean  logic  (and,  or,  noth 

A  natural  language  system  is  appropriate  in  situations  in  which  individuals 
need  to  perform  tasks  on  computers  but  nave  neither  the  desire  nor  the  time  to  learn 

I)  :! 


complex  command  languages.  Natural  language  systems  generally  have  their 
greatest  utility  in  permitting  senior  management  personnel  to  retrieve  information 
from  a  database.  Those  individuals  typically  do  not  have  the  time  to  learn  database 
query  languages  or  complex  command  languages. 

Another  task  characteristic  that  points  toward  a  natural  language  system  is  the 
method  required  to  access  information.  If  the  prospective  users  need  to  access 
information  in  an  ad  hoc  manner,  then  the  application  is  probably  best  addressed  by  a 
natural  language  system.  Ad  hoc  information  retrieval  exists  when  information 
must  be  retrieved  from  a  database  for  which  any  two  queries  and  display  formats  are 
rarely  alike.  Managers  typically  make  the  most  frequent  use  of  ad  hoc  information 
retrieval.  If  the  prospective  users  generally  retrieve  and  display  information  in  a 
relatively  few  standard  ways,  their  needs  are  best  served  using  a  menu  system. 
Natural  language  systems  are  also  useful  for  analysis  of  message  trafTic.  These 
systems  can  fiag  critical  messages  during  periods  of  intense  activity  associated  with 
war  or  emergency  actions. 

Speech  Recognition/UncJerstanding 

The  criteria  for  determining  whether  a  speech  recognition  system  is  possible  are 
similar  to  the  criteria  for  a  natural  language  system.  However,  a  natural  language 
system  processes  text,  whereas  a  speech  system  processes  spoken  words.  The 
vocabulary  of  a  speech  system  must  be  fairly  small,  limited  to  a  few  hundred  words  at 
most.  If  there  is  a  requirement  that  the  system  be  able  to  handle  multiple  users,  the 
size  of  the  vocabulary  for  each  user  must  be  even  further  reduced.  If  the  system  is 
required  to  be  independent  of  the  speaker,  the  vocabulary  must  be  limited  to  20  to 
30  words.  Further,  the  word  structure  should  be  restricted  to  one-  to-three  word 
phrases.  Current  speech  recognition  systems  are  not  able  to  deal  well  with 
continuous  speech,  a  limitation  that  will  probably  be  eliminated  within  the 
next  2  to  3  years. 

The  final  criterion  for  a  speech  system  is  that  the  background  noise  must  be 
controllable.  Hence,  the  system  must  either  operate  in  a  relatively  noise-free 
environment  or  must  have  a  means  (i.e.,  noi'=.  limiting  or  directional  microphone)  to 
reduce  the  background  noise. 

A  speech  recognition  system  is  appropriate  in  any  environment  in  which  the 
users  must  either  have  their  hands  free  to  perform  other  tasks  or  must  be  able  to 
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move  about  freely.  Some  examples  of  locations  at  which  users  must  have  their  hands 
free  to  perform  other  tasks  are  vehicle  or  aircraft  maintenance.  Aircraft  inspection 
after  periodic  depot  maintenance  is  an  activity  in  which  the  user  of  an  expert 
maintenance  system  might  need  an  integrated  speech  recognition  system  to  allow  the 
technician  to  communicate  with  the  KBS  while  moving  about  the  aircraft. 

Three-Dimensional  Vision 

The  current  state  of  the  art  in  three-dimensional  vision  is  practical  for  only  very 
simple  objects.  Current  vision  systems  are  effective  for  applications  that  involve 
recognizing  regular  shapes  (i.e.,  squares,  rectangles,  triangles,  etc.)  and  straight 
lines.  If  the  application  requires  that  the  three-dimensional  vision  system  be  able  to 
recognize  the  relative  positions  of  objects,  then  the  application  is  one  best  left  to  the 
future.  Current  three-dimensional  vision  systems  capable  of  recognizing  objects  and 
relative  position  are  extremely  slow.  They  sometimes  require  the  power  of  a 
supercomputer  for  several  hours  just  to  process  one  scene. 

Unfortunately,  given  the  current  level  of  technology,  no  guidelines  are 
available  to  indicate  when  a  three-dimensional  vision  system  might  be  appropriate. 
The  few  successful  systems  in  use  today  are  predominantly  used  in  custom  plate 
welding.  Some  future  applications  of  three-dimensional  vision  might  allow  robots  to 
visually  acquire  objects  or  might  be  used  in  aircraft/vehicle  inspection/cleaning, 
remote  sensing,  target  acquisition  and  tracking,  and  nondestructive  inspection 
activities  such  as  X-ray  analysis. 

Intelligent  Robotics 

Intelligent  robotics  is  the  least  mature  AI  technology  discussed  in  this  study. 
An  intelligent  robot  depends  on  many  of  the  other  AI  technologies  working  in  concert. 
It  requires  KBS  to  provide  the  decision-making  capability  and  control,  three- 
dimensional  vision  to  perceive  its  world,  and  mechanical  manipulators  to  enable  it  to 
alter  objects  in  the  real  world. 

To  determine  whether  an  intelligent  robotic  system  is  feasible,  a  KBS  and  a 
three-dimensional  vision  system  capable  of  performing  the  reasoning  and  perception 
functions  of  the  robot  must  first  be  possible.  Next,  the  environment  in  which  the 
system  will  operate  must  be  constrained  in  order  to  minimize  the  number  of  random 
events  occurring  around  the  robot.  This  will  reduce  the  amount  of  processing 


required  and  minimize  the  likelihood  that  the  robot  will  encounter  an  unprogrammed 
situation.  One  means  of  constraining  and  controlling  the  amount  of  uncertainty  in 
the  environment  is  to  provide  the  robot  clearly  marked  paths  or  areas  within  which  to 
operate.  Humans  and  other  objects  are  then  restricted  from  this  area.  Another 
means  of  reducing  random  events  in  the  robot’s  world  is  to  narrow  the  robot’s  scope  of 
operation  to  a  specific  task  in  which  it  is  unlikely  any  random  events  will  occur. 
Limitation  of  the  scope  of  operation  is  common  on  assembly  lines  where  robots  are 
used  for  visual  inspection  of  complex  parts. 

An  intelligent  robot  is  appropriate  in  situations  in  which  it  is  uneconomical  or 
hazardous  to  place  humans.  These  situations  include:  nuclear  reactor  inspection  and 
equipment  repair  in  a  chemical,  biological,  or  radiological  (CBR)  environment.  An 
intelligent  robotic  system  may  also  be  appropriate  in  highly  repetitive  situations  in 
which  humans  become  bored  or  in  which  a  high  degree  of  precision  and  consistency  is 
required.  These  situations  include  visual  parts  inspection  and  nondestructive 
inspection  (NDI)  of  structures.  Many  repetitive  quality  assurance  inspections  that 
require  inspection  of  a  product  and  either  the  disposition  or  repair  of  the  product  may 
also  be  appropriate  candidates  for  an  intelligent  robotic  solution. 

Neural  Network 

Neural  networks  have  been  in  existence  for  more  than  20  years,  but  only 
recently  have  they  been  applied  to  real-world  problems.  The  current  level  of 
technology  in  neural  networks  makes  them  adequate  for  fairly  small  problems. 
Neural  networks  are  essentially  pattern  recognizers  that  are  sometimes  linked  to 
vision  systems.  Tasks  that  require  the  recognition  of  recurring  patterns  are 
generally  candidates  for  neural  networks. 

Development  tools  based  on  neural  networks  are  sometimes  used  to  implement 
KBS.  When  that  occurs,  the  criteria  for  KBS  should  be  examined  to  determine 
whether  the  application  is  possible  and  appropriate.  Selecting  a  neural-network- 
based  tool  over  a  more  conventional  rule-and-frame-based  tool  is  primarily  a  matter 
of  personal  preference.  The  one  criterion  for  neural  networks  that  differs  from  a 
criterion  for  KBS,  in  general,  is  the  latter’s  requirement  for  a  human  expert.  Since 
neural  networks  are  capable  of  discovering  patterns  in  data  independent  of  a  human 
expert,  they  are  not  as  reliant  on  human  knowledge  for  development  as  are 
conventional  KBS.  However,  the  question  of  performance  validation  remains. 


Without  a  human  expert,  the  performance  of  a  neural  network  application  may  be 
difficult  to  validate.  For  that  reason,  anyone  attempting  to  build  a  KBS  using  a 
neural  network  in  the  absence  of  a  human  expert  is  strongly  advised  to  proceed 
cautiously.  Other  than  being  somewhat  less  reliant  on  a  human  expert  than 
conventional  KBS,  the  advantages  of  current  neural-network-based  knowledge  tools 
over  conventional  tools  are  nebulous  at  best. 

The  criteria  for  neural  networks  are  essentially  the  same  as  for  KBS. 
Frequently  neural  networks  serve  simply  as  another  knowledge  representation 
methodology,  much  like  rules,  frames  and  semantic  networks.  However,  neural 
networks  are  very  good  at  discovering  and  recognizing  patterns  in  large  databases. 

JUSTIFYING  AN  ARTIFICIAL  INTELLIGENCE  SOLUTION 

In  justifying  an  AI  solution,  operational  effectiveness  and  economic 
considerations  must  be  weighed.  Operational  effectiveness  involves  retaining  crucial 
expertise  and  having  it  available  at  all  times,  in  all  locations,  and  under  all 
conditions.  Expertise  critical  to  a  mission  may  be  lost  through  the  retirement  of 
experts  before  new  workers  can  develop  the  expertise. 

In  another  case,  the  expert  may  be  promoted  to  a  position  that  requires 
management  talent,  for  example,  rather  than  technical  expertise.  The  expertise 
remains  with  the  organization  but  not  where  needed.  This  latter  problem  occurs 
often  in  DoD.  To  be  promoted,  DoD  personnel  in  scientific  and  engineering 
disciplines  must  move  to  management  positions.  Thus,  their  technical  expertise  is 
effectively  lost  to  the  organization. 

In  yet  a  third  case,  a  critical  expertise  may  simply  be  scarce.  For  a  variety  of 
reasons  the  organization  may  have  a  difficult  time  recruiting  personnel  with  certain 
capabilities,  or  the  expertise  may  be  needed  in  many  locations  or  in  locations  at 
which  it  is  impractical  or  impossible  to  place  human  beings. 

In  addition  to  operational  effectiveness,  Al  solutions  should  be  justified  on  the 
basis  of  economics.  The  cost  savings  produced  by  using  the  system  should  outweigh 
the  cost  of  developing,  fielding,  and  maintaining  the  AI  system.  The  factors  that 
must  be  considered  when  determining  the  return  on  investment  (ROD  for  an  AI 
system  are  addressed  in  the  next  section. 


EVALUATING  ARTIFICIAL  INTELLIGENCE  PROJECTS 


The  Services  should  use  a  standard  payback  model  for  assessing  the  payback 
resulting  from  the  application  of  AI  technologies.  The  same  standard  model  can  be 
applied  to  all  of  the  AI  technologies  discussed  in  this  report.  Use  of  a  standard 
payback  model  has  many  advantages.  First,  it  ensures  organizations  submitting 
projects  consider  all  the  salient  factors  when  computing  the  costs  and  benefits  of  AI 
applications.  In  the  past,  organizations  not  using  a  standard  payback  model 
generally  underestimated  the  cost  of  maintaining  software,  if  they  even  considered 
software  maintenance.  Use  of  a  standard  model  will  ensure  everyone  estimates  costs 
and  benefits  in  a  consistent  manner.  Obviously,  the  degree  to  which  the  life  cycle 
cost  (LCC)  analysis  is  formalized  will  depend  upon  the  size  of  the  AI  system  being 
developed.  It  would  not  be  appropriate  to  spend  more  time  and  money  justifying  an 
AI  system  than  was  invested  in  the  development  of  the  system. 

The  standard  model  should  consider  the  total  LCC  of  the  system.  The  LCC  of 
the  AI  system  comprises  the  development,  fielding,  and  maintenance  costs.  Each  of 
those  costs  is  examined  in  turn  in  the  following  discussion. 

DEVELOPMENT  COSTS 

Development  Hardware  Costs 

The  development  hardware  costs  include  all  the  hardware  necessary  to  develop 
the  primary  computer  software  (i.e.,  the  primary  computer,  printers,  worksta¬ 
tions/terminals,  etc.)  as  well  as  any  support  hardware  (local  area  networks,  modems, 
communication  lines,  etc.)  necessary  to  develop  the  AI  application. 

Development  Software  Costs 

The  development  software  includes  the  software,  such  as  compilers,  operating 
system,  and  KBS  development  tools  necessary  to  develop  the  application  software. 
The  development  software  costs  also  include  the  costs  of  developing  the  user  and 
maintenance  documentation  and  the  documentation  software  (i.e.,  word  processor, 
desktop  publishing  software,  etc.) 

Development  Personnel  Costs 

The  development  personnel  costs  include  the  costs  of  the  expert,  knowledge 
engineer,  software  programmer,  policy  representative,  and  any  additional  personnel 


required  to  develop  the  AJ  application.  Those  costs  are  generally  quoted  as  the 
individual  salaries  plus  the  costs  of  overhead  required  to  support  the  personnel. 

Time  to  Complete  Development 

The  time  to  complete  development  is  the  amount  of  time,  generally  in  years  and 
fractions  of  years,  from  inception  of  the  development  effort  until  the  system  is 
declared  ready  for  operational  use. 

Total  Development  Costs 

Total  development  costs  =  hardware  costs  +  software  costs 
+  (personnel  costs  X  development  time). 

FIELDING  COSTS 
Fielding  Hardware  Costs 

The  fielding  hardware  costs  include  any  additional  computer  hardware 
necessary  to  allow  the  AI  application  users  to  operate  the  application. 

Costs  of  Run-Time  Software 

Most  AI  development  tools  have  a  fee  associated  with  delivery  and  operation  of 
AI  applications  developed  using  the  tool.  A  cost  may  also  be  associated  with 
delivering  the  supporting  database  management  tools. 

Distribution  and  Publishing  Costs 

The  distribution  and  publishing  costs  are  the  costs  associated  with  producing 
the  required  number  of  copies  of  the  AI  application  disks  and  the  user  and 
maintenance  documentation. 

Number  of  User  Sites 

The  number  of  user  sites  is  determined  by  the  licensing  agreement  associated 
with  delivery  of  the  AI  development  tool.  Some  agreements  define  a  site  as  an 
individual  workstation  while  others  define  a  site  as  a  building  or  a  mainframe 
computer. 
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Total  Fielding  Costs 

Total  fielding  costs  —  hardware  cost  +  ( runtime  costs  X  number  of  sites) 

+  distribution  and  publishing  costs. 

MAINTENANCE  COSTS 

Maintenance  costs  of  software  are  extremely  difficult  to  determine  prior  to  any 
maintenance  history  for  the  application.  AI  system  software  should  have  lower 
maintenance  costs  than  that  of  conventional  systems  because  the  knowledge  that  the 
AI  program  contains  has  been  separated  from  the  logic  of  how  the  program  uses  its 
knowledge.  Thus,  maintenance  cost  savings  are  frequently  counterbalanced  by  the 
enormous  maintenance  enhancement  efforts  that  the  use  of  expert  systems  creates. 
Industry  studies  indicate  that  conventional  software  maintenance  costs  account  for 
as  much  as  50  percent  of  an  organization’s  total  Automatic  Data  Processing  (ADP) 
budget  [D-1].  More  recent  studies  by  the  Electronic  Industries  Association  (EIA) 
indicate  that  the  maintenance  cost  of  a  software  application  nia>  run  as  high  as 
60  percent  to  70  percent  of  the  LCC  of  the  application.  That  estimate  means  that 
maintaining  software  costs  2  to  2  1/2  times  the  cost  of  developing  the  software. 

Maintenance  Hardware  Costs 

The  maintenance  hardware  costs  include  all  the  hardware  necessary  to  perform 
periodic  updates  and  error  corrections  for  an  AI  application.  Many  times  a  set  of 
maintenance  hardware  can  be  us<^d  to  maintain  more  than  one  AI  application.  When 
that  is  the  case,  only  the  percentage  of  hardware  cost  associated  with  maintaining  a 
given  application  should  be  used. 

Maintenance  Software  Costs 

The  maintenance  software  costs  include  the  cost  of  the  compilers,  AI 
development  tools,  and  other  software  necessary  to  perform  periodic  updates  and 
error  corrections  for  the  AI  application. 

Maintenance  Personnel  Costs 

The  maintenance  personnel  costs  are  the  costs  of  the  expert,  knowledge 
engineer,  conventional  programmers,  policy  representatives,  and  any  other 
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personnel  associated  with  maintaining  the  software.  The  costs  should  include  the 
personnel  salaries  and  the  overhead  costs  associated  with  the  personnel. 

Distribution  and  Publication  Costs 

These  costs  are  the  costs  associated  with  distributing  the  software  updates  and 
the  revisions  to  the  user  manuals  and  maintenance  manuals. 

Expected  Life  of  the  System 

The  expected  life  of  the  system  is  the  period  of  time  from  completion  of 
development  until  the  application  has  outlived  its  usefulness  or  is  superseded  by  a 
newer  application. 

Total  Maintenance  Costs 

Total  maintenance  costs  =  hardware  costs  +  software  costs 
-h  ( personnel  costs  X  expected  life) 

+  distribution  and  publishing  costs. 

LIFE-CYCLE  COST 

The  total  LCC  of  the  system  is  the  sum  of  the  development,  fielding,  and 
maintenance  costs. 

IDENTIFICATION  OF  BENEFITS 

The  standard  model  should  consider  benefits  in  the  three  areas  described  in  the 
following  subsections. 

Benefits  Derived  from  Increased  Performance 

Benefits  derived  from  increased  performance  can  generally  be  quantified  as 
reduced  manpower  costs,  more  components  produced  per  hour,  manhour  savings,  or 
similar  factors.  The  value  of  the  increased  number  of  components  or  manhour 
reductions  is  the  benefit  derived  from  increased  performance. 

Benefits  Derived  from  Reduced  Errors 

Benefits  derived  from  reduced  errors  are  generally  more  difficult  to  quantify. 
They  include  reducing  the  cost  to  correct  an  error  (in  manhours)  or  reducing  the  cost 
to  identify  and  replace  a  faulty  component  that  was  incorrectly  repaired.  A  major 
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factor  that  affects  the  beriefits  derived  from  reduced  errors  is  an  accurate  estimation 
of  the  error  rate  prior  to  implementing  the  AI  system  and  the  anticipated  error  rate 
after  implementing  it.  The  difference  between  those  two  figures  times  the  cost  of 
fixing  the  errors  yields  the  total  benefits  derived  from  reduced  errors. 

Benefits  Derived  from  Increased  Consistency 

These  benefits  are  extremely  difficult  to  quantify  and  are  generally  assessed 
best  by  higher  command  levels. 

SELECTION  OF  PAYBACK  MODEL 

To  simplify  the  process  of  determining  payback  on  an  AI  application,  and  since 
most  AI  applications  will  be  part  of  larger  conventional  data  systems  and  weapon 
systems,  the  payback  model  used  should  be  compatible  with  the  funding  source  of  the 
system  of  which  the  AI  application  is  a  part.  If  the  AI  application  is  stand-alone  and 
the  developers  request  funding  through  an  established  funding  source  [such  as 
Pr iducti"'*::’  Investment  Funding  (PIF)],  then  the  payback  model  is  usually 
mandated  by  the  existing  funding  source. 

If  the  AI  application  developers  request  funds  from  a  source  established 
expressly  for  AI  applications,  the  payback  model  used  should  be  the  one  described 
herein.  Whatever  funding  source  is  used,  the  organizations  evaluating  the  requests 
for  funds  should  be  aware  of  the  issues  involved  in  AI  applications  and  should  be 
aware  of  the  intangible  benefits  as  well  as  the  pitfalls  of  the  AI  applications. 

.ETTING  PRIORITIES  FOR  AI  PROJECTS 

Priorities  for  proposed  AI  projects  should  be  set  in  two  steps.  The  first  step  is  to 
apply  the  selection  criteria  discussed  above  to  determine  the  appropriateness  of  AI  for 
the  projects.  If  a  project  is  determined  to  be  appropriate  for  use  of  AI  technology,  it 
passes  to  the  second  step  in  the  process.  In  the  second  step,  the  relative  worth  of  the 
project  is  determined  and  the  priority  is  established. 

Project  Screening 

Project  screening  is  used  simply  as  a  filter  to  aid  in  increasing  the  probability  of 
success  of  the  projects  that  are  eventually  funded.  Each  project  submitted  is  subject 
to  the  criteria  described  above  depending  on  the  specific  AI  technology  involved.  In 
the  event  that  several  AI  technologies  are  involved,  the  project  subcomponents  are 
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subjected  to  the  criteria  described  above,  and  only  if  each  subcomponent  is 
appropriate  will  the  entire  project  be  deemed  appropriate. 

This  initial  screening  will  determine  whether  an  AI  solution  is  possible  for  each 
nominated  project  and  will  also  determine  whether  the  technology  is  appropriate. 
This  will  increase  the  likelihood  that  the  projects  will  be  successfully  completed. 

Return  on  Investment 

Priorities  for  the  proposed  projects  are  really  established  as  part  of  the  ROI 
evaluation.  Priorities  should  be  set  only  for  those  projects  that  are  deemed  to  be 
appropriate  in  the  project  screening  step.  Furthermore,  every  project  that  enters  this 
second  step  does  so  on  an  equal  footing.  Since  the  projects  are  evaluated  in  Step  1 
using  different  selection  criteria  based  on  the  technology  involved,  we  cannot 
determine  the  relative  value  of  projects.  To  try  to  set  priorities  for  projects  based  on 
the  outcome  of  the  project  screening  would  be  like  trying  to  say  apples  are  better  than 
oranges. 

The  ROI  determination  should  look  at  two  primary  factors.  The  first  factor  is 
the  tangible  value  of  the  project,  that  is,  how  much  money  the  project  will  generate  or 
save.  The  second  factor  is  the  intangible  contribution  to  the  using  organization’s 
mission  effectiveness,  that  is,  how  much  will  the  project  contribute  to  national 
security.  Since  the  relative  contribution  to  mission  effectiveness  is  difficult  to 
quantify,  the  projects  should  first  be  ranked  based  upon  the  relatively  tangible  ROI. 
The  intangible  benefits  should  be  used  only  in  the  case  of  a  tie  should  two  projects 
have  the  same  tangible  ROI. 

Interoperability 

When  developing  interoperable  or  common  AI  applications,  three  system 
components  need  to  be  considered:  the  user  interface,  the  inference  engine,  and  the 
domain  knowledge.  Developing  AI  systems  with  common  user  interfaces  and 
inference  engines  greatly  reduces  the  amount  of  time  it  takes  system  developers  and 
users  to  adapt  to  new  AI  systems.  The  benefits  expected  from  a  common  user 
interface  and  inference  engine  would  be  analogous  to  the  benefits  anticipated  from 
developing  conventional  software  in  a  common  programming  language  like  Ada.  We 
strongly  recommend  that  AI  developers  adopt  common  user  interfaces  and  inference 
engines.  They  can  do  so  in  the  same  way  that  DoD  adopts  approved  higher-order¬ 
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languages  (HOLs)  for  conventional  software.  None  of  the  current  AI  development 
tools  is  on  the  list  of  approved  HOLs  for  defense  system  development.  While  existing 
HOLs,  like  Ada,  may  be  sufficient  for  conventional  software  and  may  to  some  extent 
be  used  to  implement  simple  AI  systems,  they  are  neither  adequate  nor  efficient  for 
developing  most  real-life  AI  applications.  DoD  needs  to  determine  whether  it  desires 
to  add  existing  development  tools  to  the  list  of  approved  HOLs  or  if,  like  the  Ada 
project,  it  wishes  to  develop  a  DoD  proprietary  tool  for  AI  development.  Adding  AI 
development  tools  to  the  list  of  approved  HOLs  will  aid  in  reducing  the  proliferation 
of  different  tools.  The  approval  process  for  nonstandard  tools  should  be  the  same  as 
for  other  nonstandard  HOLs. 

Developing  AI  systems  with  common  domain  knowledge  is  difficult.  Building 
specific  domain  applications  that  can  be  used  across  many  functional  areas  is  very 
difficult,  and  no  definitive  rules  have  been  established  for  determining  which 
applications  are  appropriate.  One  way  that  multiple  organizations  can  develop  AI 
systems  is  to  have  a  central  organization  monitor  all  AI  development  efforts  and 
facilitate  the  free  exchange  of  ideas  among  the  different  AI  development 
organizations.  The  central  management  organization  can  also  form  a  cadre  of 
development  personnel  that  can  assume  development  responsibilities  for  projects 
that  span  a  large  number  of  functional  areas  or  services. 

We  recommend  that  wherever  possible,  DoD  develop  AI  systems  that  can  be 
used  across  many  functional  areas.  While  that  approach  may  increase  the  initial 
development  cost  for  some  applications  since  the  requirements  of  multiple 
organizations  will  have  to  be  taken  into  account,  the  ROI  should  be  higher  because  of 
the  increased  user  base.  Decision  makers  at  all  levels  must  be  aware  that  the  narrow 
application  domain  of  AI  systems  makes  it  difficult  to  use  seemingly  similar  appli¬ 
cations  across  functional  boundaries. 
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APPENDIX  E 


THE  MANAGEMENT  OF  ARTIFICIAL  INTELLIGENCE  PROJECTS 

DEVELOPING  AND  ACQUIRING  SYSTEMS 

Artificial  intelligence  (AJ)  systems  are  developed  to  solve  difTicult  problems 
whose  solution  process  is  subjective  and  frequently  not  well  understood.  Because  of 
the  nature  of  AI  technology,  development  and  acquisition  documents,  such  as  the 
system  specification,  appear  somewhat  later  in  the  development  cycle  of  AI  software 
than  they  do  in  the  conventional  software  development  cycle.  For  that  reason,  it  may 
be  desirable  to  begin  development  of  many  AI  systems  in-house.  In-house  li.e., 
organic)  development  can  serve  to  prove  the  concept  and  can  determine  the 
appropriate  scope  of  the  application.  Once  the  concept  is  proven  and  its  scope  is 
defined,  the  developers  can  decide  whether  to  continue  the  development  in-house  or 
to  do  so  under  contract. 

Location  of  Development  Capabilities 

The  issue  of  where  the  development  capabilities  for  AI  projects  should  reside  is 
complex.  To  develop  an  efTective  AI  application,  the  developer  must  have  detailed 
knowledge  of  the  application  domain,  which  would  indicate  that  the  development 
capability  should  reside  with  the  users.  However,  many  applications  apply  across 
organizational  boundaries  and  require  a  detailed  knowledge  of  the  overall 
application  domain  and  a  broad  vision  of  where  the  applications  will  be  used  and  by 
whom.  Those  more-sophisticated  applications  would  require  a  centralized  devel¬ 
opment  facility  above  the  user  level. 

Most  organizations  and  companies  discussed  in  Chapter  2  of  this  report  provide 
for  both  ?  decentralized  and  a  centralized  development  capability.  The  primary 
development  capabilities  should  reside  with  the  users  because  they  have  the  detailed 
domain  knowledge  that  makes  successful  AI  systems  possible.  However,  a  central 
organization  needs  to  monitor  the  development  activities  to  ensure  cross-flow  of 
information  and  to  determine  which,  if  any,  of  the  development  efforts  will  lead  to  a 
system  that  has  applicability  across  organizational  boundaries.  When  a  system  that 
can  be  used  by  many  organizations  is  identified,  the  central  organization  needs  to 
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have  the  authority  to  direct  the  developing  organization  to  work  with  the  other 
potential  users  to  ensure  everyone’s  requirements  are  incorporated  into  the  system’s 
design.  The  central  organization  must  have  some  capability  to  enable  it  to  either 
develop  or  contract  those  systems  that  have  application  across  many  organizational 
lines  but  for  which  no  developers  at  the  user  level  have  been  found. 

Appropriate  Acquisition  Strategy 

Acquisition  personnel  should  be  aware  that  the  tasks  performed  by  AI  software 
are  an  order  of  magnitude  more  complex  than  those  performed  by  conventional 
automatic  data  processing  ( ADP)  or  mission-critical  software.  A  common  technique, 
employed  successfully  for  more  than  10  years  to  rapidly  develop  AI  software,  is 
known  as  rapid  prototyping.  Rapid  prototyping  begins  with  a  simple  statement  of 
requirements  and  uses  the  process  of  developing  the  software  to  generate  and  refine 
the  system  specifications.  Only  small,  well-defined  tasks  are  prototyped,  and  the 
system  evolves  as  the  t-asks  performed  by  each  successive  prototype  encompass  more 
and  more  of  the  total  system  functional  requirement.  A.  key  component  of  rapid 
prototyping  is  the  shell  or  tool  used  to  develop  the  software.  Developers  using  rapid 
prototyping  techniques  do  not  try  to  program  directly  in  a  higher  order  language 
iHOL  i  such  as  Ada.  FORTRAN,  or  LISP.  Rather,  they  use  high-level  programming 
tools  or  environments  such  as  the  Knowledge  Engineering  Environment  (KEE), 
Smalltalk,  or  the  numerous  LISP  programming  environments.  By  using  those 
environments,  software  developers  are  able  to  achieve  10-to-l  productivity  increases 
over  conventional  programming  techniques.  Hence,  rapid  prototyping  permits 
system  replacement  leadtimes  to  be  considerably  reduced  from  leadtimes  of 
conventional  software  development  efTorts. 

Organizations  that  plan  to  use  AI  to  solve  a  problem  should  first  develop  a  small 
prototype  on  a  well-constrained  part  of  the  overall  task.  That  process  will  enable  the 
organization  to  better  prepare  detailed  specifications  for  the  full  system.  The 
prototype  along  with  the  specifications  should  then  be  provided  to  the  final  system 
developers  as  Government  Furnished  Equipment  (GFE).  "Whether  the  system 
developer  is  another  DoD  organization  or  a  contractor,  the  functioning  prototype 
combined  with  the  system  specification  will  greatly  aid  the  final  system  development 
activity. 


No  special  review  process  is  required  for  AJ  projects.  The  decision  thresholds  in 
place  for  the  acquisition  of  existing  systems  should  also  be  used  for  AI  systems. 
Howeve.*,  organizations  acquiring  AI  systems  should  ensure  there  is  a  continuous 
exchange  of  information  between  the  system  developers  and  the  users.  In  contract 
situations  this  exchange  will  mean  greater  user  involvement  than  may  have  been 
usual  in  conventional  software  development.  It  will  also  mean  that  the  acquisition 
office  will  have  to  conduct  more  frequent  technical  interchange  meetings  with  the 
system  developers  and  more  frequent  design  reviews. 

An  effective  baselining  and  configuration  control  process  is  required  for  .Xl 
systems.  It  is  impossible  to  complete  and  field  an  accurate  and  reliable  systo.m 
without  this  process.  DoD  already  has  an  effective  baselining  and  configuration 
control  system  described  in  the  two  software  development  standards. 
DoD-STD-2167A,  Defense  System  Software  Development  S tandard.  and  DoD- 
STD'2168,  Defense  System  Software  Quality  Program  Standard.  Although  these 
standards  were  established  for  conventional  software,  they  are  also  applicable  to  AI 
software.  Using  these  existing  development  and  quality  standards  provides  the 
added  benefit  of  implementing  a  consistent  software  management  policy  across  ni! 
types  of  software  used  in  DoD. 

Major  unresolved  issues  still  exist  in  contracting  for  AI  systems.  Currently,  no 
standards  govern  AI  development  tools.  DoD  has  established  a  list  of  approved 
programming  languages  for  conventional  software  development,  but  to  date,  it  na.s 
identified  no  standard  AI  development  tools.  DoD  now  has  a  prime  opportunity  t*' 
take  the  lead  and  establish  a  set  of  standard  AI  system  development  tools  to  avoid  thi* 
proliferation  of  development  tools  experienced  in  conventional  software.  The  Defense 
Advanced  Research  Projects  Agency  is  currently  working  on  developing  an  expert 
system  development  tool,  ABE,  that  may  establish  a  baseline  from  which  to  begin 
establishing  standard  AI  development  tools. 

It  remains  to  be  seen  what  the  best  contracting  methodologies  for  .AI  systems 
are.  For  small,  well-defined  systems,  firm  fixed  price  (FFP)  contracc.s  may  be  the  best 
types  to  use.  However,  for  larger,  less- well-defined  AI  applications,  contract«^rs  mav 
be  unwilling  to  accept  the  risk  associated  with  a  FFP  I’ontract  and  will  insist  on  co^t 
plus  contracts.  Cost  plus  contracts  put  a  larger  share  of  the  risk  on  the  Government. 
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especially  in  situations  in  which  rapid  prototyping  begins  without  detailed 
specifications. 


DoD  should  establish  policies  concerning  standard  AI  development  tools  and 
contracting  methodologies.  It  should  examine  how  contractual  obligations  regarding 
performance  of  AI  systems  should  be  handled  in  statements  of  work  (SOW)  and 
specifications. 

FIELDING  SYSTEMS 

Documentation  Required  for  the  Maintenance  of  AI  Systems 

AI  software  must  be  documented  to  enable  system  users  and  maintainers  to 
effectively  perform  their  respective  functions.  In  1987,  the  Electronic  Industries 
Association  (EIA)  established  a  subgroup  on  the  Management  of  Artificial 
Intelligence  Software  to  address  this  issue.  The  committee’s  recommendations  were 
documented  in  the  EIA  G-33  and  G-34  Committee  Proceedings.  The  group  concluded 
that  the  documentation  requirements  described  in  DoD-STD-2167A  were  adequate 
for  AI  software  provided  that  they  were  appropriately  tailored.  The  committee’s 
report  describes  suggested  tailoring  for  several  of  the  data  item  descriptions  (DIDs) 
associated  with  DoD-STD-2167A. 

The  specific  documentation  required  for  an  AI  system  varies  with  the  size, 
complexity,  critical  nature  of  the  application,  method  of  procurement,  and  cost  of  the 
application.  As  the  size,  complexity,  and  cost  of  an  application  increases,  the 
organization  acquiring  the  system  will  wish  to  have  greater  insight  into  the  actual 
development  process.  The  procurement  method  may  also  have  an  impact  on  the 
amount  of  documentation  required.  In  general,  if  the  application  is  being  developed 
using  in-house  resources,  documentation  is  not  required  to  the  same  level  of  detail  as 
is  required  when  the  application  is  developed  by  an  outside  agency.  These  issues  are 
fully  discussed  in  DoD-HDBK-287,  .4  Tailoring  Guide  for  DoD-STD-2167A.  Defense 
System  Software  Development.  An  expert  system  developed  by  the  Air  Force 
Acquisition  Logistics  Center,  the  Software  Support  Decision  Aid,  also  identifies 
software  documentation  requirements. 

As  a  minimum,  every  AI  system  developed  should  have  a  Software  User’s 
Manual,  a  Software  Programmer’s  Manual,  and  a  Software  Design  Document.  The 
Software  User’s  Manual  provides  the  instructions  necessary  to  allow  the  functional 
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users  to  operate  the  system.  The  Software  Programmer’s  Manual,  in  many  cases, 
consists  of  the  commercial-off-the-shelf  documentation  that  describes  the  operation 
and  use  of  the  AI  tool  used  to  develop  the  AJ  application.  When  the  AI  application  is 
developed  with  other  than  an  AJ  development  tool,  the  Software  Programmer’s 
Manual  should  describe  the  operation  and  use  of  the  programming  language  and 
environment,  if  any,  used  to  develop  the  AI  application.  The  Software  Design 
Document  describes  in  detail  the  structure  and  operation  of  the  AI  software.  The 
Software  Design  Document  should  contain  information  on  the  physical  structure  of 
the  AI  software  as  well  as  all  process  flows.  The  complete  content  of  these  documents 
is  described  in  the  DIDs,  DI-MCCR-80012A,  DI-MCCR-80019A,  and  DI-MCCR- 
80021A. 

Testing  Required  Prior  to  Fielding 

Verification  and  validation  must  be  considered  when  testing  AI  software. 
Verification  is  the  set  of  procedures  used  to  determine  the  correctness  of  the 
knowledge  contained  in  the  AI  system;  validation  determines  that  the  system 
satisfies  the  original  statement  of  the  requirement.  Both  should  be  performed  on 
every  system  developed.  However,  for  small  systems  which  have  relatively  short 
development  times  and  relatively  low  development  costs,  validation  may  not  be  cost- 
effective.  Eliminating  the  validation  will  reduce  the  cost  of  implementing  the 
system.  For  systems  with  short  development  periods.  (3  to  6  man  months),  the 
developers  are  not  likely  to  stray  from  the  original  system  requirements  and  the 
validation  step  may  usually  be  safely  eliminated.  Rapid  prototyping  also  increases 
the  likelihood  that  the  system  will  satisfy  the  original  requirements  without 
additional  validation. 

AI  system  verification  must  be  performed.  Verification  is  the  only  way  to 
determine  that  the  advice  given  by  the  AI  system  is  correct  and  accurate.  Since  by 
their  very  nature  AI  systems  do  not  conform  to  numeric  algorithms,  that  when 
correct,  are  certain  to  give  correct  answers,  their  correctness  cannot  be  proved,  as  is 
theoretically  possible  for  conventional  software.  The  usual  method  of  performing 
system  verification  is  to  have  the  experts  review  the  knowledge  base  entries  for 
consistency  and  accuracy. 


Al  VALIDATION  TECHNIQUES 


The  two  techniques  used  to  validate  the  accuracy  of  AI  systems  are  the 
testimonial  and  the  statistical  methods.  Whether  the  statistical  method  or 
testimonial  method  of  evaluating  AI  systems  is  used,  clear  and  precise  evaluation 
criteria  must  be  established  at  the  time  the  system  specifications  are  being 
developed. 

When  the  functional  requirements  for  the  AI  system  are  established,  the 
criteria  for  an  acceptable  system  must  also  be  established.  Test  data  (or  cases)  must 
be  selected  and  a  final  qualification  test  must  be  performed  prior  to  system 
acceptance.  For  example,  a  criterion  might  be  that  the  system  be  able  to  process 
100  test  cases  in  a  given  amount  of  time  and  to  provide  100  percent  (or  less  if  desired) 
correct  recommendations. 

Testimonial  Method 

The  testimonial  method  is  chosen  when  no  test  data  are  available  with  which  to 
objectively  measure  the  advice  of  the  AI  system.  Such  a  situation  occurs  frequently 
with  AI  systems.  The  testimonial  validation  presents  the  AI  system  with  a  set  of  test 
cases  to  solve.  The  "solved”  test  cases  are  then  provided  to  an  expert  or  panel  of 
experts  for  evaluation.  The  reviewing  experts  determine  what  percentage  of  the  test 
cases  are  correct.  If  the  percentage  of  test  cases  correct  is  high  enough,  the  system 
passes  validation.  Some  examples  of  applications  for  which  this  is  the  preferred 
method  of  validation  are  contract  warranty  selection,  software  documentation 
selection,  inventory  stocking  level  determination,  and  shop  floor  scheduling.  All 
those  examples  are  applications  for  which  no  answer  is  absolutely  right  or  wrong  and 
thus  we  have  no  objective  way  to  verify  the  performance  of  the  system. 

Statistical  Method 

The  statistical  method  is  chosen  when  we  have  test  data  with  which  to 
objectively  compare  the  advice  of  the  AI  system.  The  statistical  method  requires  a  set 
of  known  correct  solutions  to  the  test  cases.  The  test  cases  are  presented  to  the  AI 
system  and  the  AI  system’s  advice  is  compared  to  the  known  correct  results.  If  the  AI 
system  achieves  a  satisfactory  number  of  correct  answers,  the  system  passes  the  test. 
Some  examples  of  applications  for  which  we  have  absolutely  correct  answers  are  fault 
isolation,  repair,  and  data  validation. 


Neither  of  these  testing  methodologies  takes  into  account  the  performance  level 
of  the  AJ  system  in  relation  to  the  performance  of  humans  executing  the  same  task. 
An  expert  system  (ES)  that  provides  correct  advice  on  45  percent  of  ail  test  cases  may 
not  appear  to  have  an  acceptable  level  of  performance.  However,  the  benefits  of  the 
ES  are  clear  when  one  realizes  that  the  average  human  performing  the  same  task 
only  gets,  for  example,  20  percent  of  the  test  cases  correct.  Therefore,  test  cases 
should  be  tested  against  both  the  AJ  system  and  a  representative  sample  of  humans. 
We  can  then  measure  the  performance  of  the  AJ  system  relative  to  that  of  humans 
doing  the  same  tasks. 

MAINTAINING  Al  SYSTEMS 

While  AJ  systems  such  as  rule-based  ES  may  be  easier  to  maintain  than 
conventional  systems  of  similar  complexity,  AJ  systems  tend  to  be  modified  more 
frequently  than  conventional  software.  This  increased  frequency  of  modification 
stems  from  two  factors.  First,  since  some  AI  software  is  easier  to  modify  than 
conventional  software,  users  tend  to  ask  for  more  frequent  changes  than  they  would  if 
they  knew  it  would  take  a  long  time  to  implement  the  change.  Second,  the  level  of 
knowledge  that  such  AJ  systems  as  ES  deal  with  is  more  familiar  to  the  users  than 
the  level  of  knowledge  in  conventional  software.  For  that  reason,  users  are  more 
aware  of  how  ES  operate  and,  therefore,  request  more  frequent  changes  to  match  the 
changes  in  how  they  do  their  jobs.  Because  of  the  frequency  of  changes  to  AJ  software 
and  the  level  of  domain  knowledge  contained  in  the  application,  domain  experts  and 
users  must  be  involved  in  maintaining  AJ  software. 

A  large  percentage  of  AJ  systems  may  be  developed  by  end  users,  and  end  users 
rarely  anticipate  the  level  of  effort  required  to  maintain  an  operational  system.  As  a 
result,  they  tend  to  overextend  their  resources.  However,  since  end  users  better 
understand  their  operational  requirements,  procedures  must  be  established  to  allow 
them  to  develop  their  own  systems  and  transfer  the  maintenance  of  those  systems  to 
a  maintenance  organization  for  long-term  support.  That  practice  will  ensure  needed 
systems  are  developed  on  a  timely  basis  while  reducing  the  software  support  burden 
on  the  end  users.  This  will  also  allow  the  different  organizations  to  concentrate  their 
AJ  system  support  personnel  and  more  effectively  allocate  these  valuable  resources. 

If  the  using  organization  is  also  the  developing  organization,  there  is  no  major 
problem  with  the  users  maintaining  their  own  systems.  However,  in  situations  in 


which  the  users  have  not  developed  the  system  or  the  system  is  used  by  many 
organizations,  the  issues  of  configuration  control  and  management  become  much 
more  critical.  In  those  situations,  the  maintenance  should  be  left  to  a  central  support 
center  with  expertise  in  software  configuration  management.  Even  those  systems 
developed  and  maintained  by  the  end  users  require  some  configuration  management 
and  should  be  maintained  by  a  single  person  or  group  within  the  user’s  organization. 

The  developers  of  any  large  system  should  consider  fielding  an  automated 
knowledge  acquisition  system  along  with  the  operational  system  so  that  the  users 
can  easily  update  the  knowledge  in  their  AI  system  without  having  to  manually  alter 
any  system  code.  Automated  knowledge  acquisition  systems  can  greatly  facilitate 
the  job  of  acquiring  the  new  knowledge  necessary  to  maintain  and  modify  an  existing 
system.  The  trade-ofT  that  must  be  examined,  when  considering  the  development  of  a 
knowledge  acquisition  system,  is  the  cost  of  developing  the  knowledge  acquisition 
system  versus  the  cost  of  manually  maintaining  the  AI  system.  Automated 
knowledge  acquisition  systems  are  economically  justified  for  only  very  large  systems. 
In  some  cases,  the  development  cost  of  the  knowledge  acquisition  system  may  equal 
the  cost  of  developing  the  operational  knowledge  system.  DoD  should  examine 
closely  the  cost  of  maintaining  a  very  large  knowledge  system.  If  that  cost  will  be 
greater  than  70  percent  to  80  percent  of  the  total  life-cycle  cost  (LCC)  of  the  system, 
the  developers  should  strongly  consider  developing  an  automated  knowledge 
acquisition  system. 


